Multi-Thread Web Simulator

Home Page > Software > Multi-Thread Web Simulator

Current Version = 1.3.0.1

MultiThreadWebSimulator is, as the name suggests, a multi-threaded web simulator. It is a command-line, Windows-based application that I wrote at home in 2006 and then brought in to work to help load-test a wonderful web-based application called "MOVEit DMZ."

During testing, MultiThreadWebSimulator reads through a list of "link macros" to figure out what it should do and each thread follows the same path through the list. However, the use of various "macros" allow each thread to use different authentication, different paths, etc. To simulate a real user environment, each session pauses a random amount of time (between X and Y seconds where X and Y are configurable). Also, to avoid a similarly unrealistic situation in which many threads/users are performing similar scripted actions at the same time, the start of each web session is staggered across the threads.

Long story short, you use MultiThreadWebSimulator like this:

  1. Build a configuration file containing the URLs and "link macros" you want to test.
  2. Run MultiThreadWebSimulator.exe using a named configuration file and several command-line options.
  3. If you see timeouts or errors in the MultiThreadWebSimulator output or errors in your web applications log, you may have found a performance limit or bugs. Analyze, fix, and repeat...
  4. If you like what you see out of steps 1-3, automate your script(s) into "smoke tests", etc...

Documentation Table of Contents

Prerequisites

Either Mono 2.0 or Microsoft .NET 2.0. This application was compiled twice: first using Microsoft Visual Studio 2005 and .NET 2.0.50727 and again using Mono 2.0.1 for Windows.

(I haven't tried this application under Mono on non-Windows machines but it uses some Windows libraries to change console colors so I'd expect trouble.)

Download

MultiThreadWebSimulator 1.3.0.1 for .NET 2.0

MultiThreadWebSimulator 1.3.0.1 for Mono 2.0 (for Windows)

(Both of these files are smaller than 30KB. On my Vista Ultimate machine, the native .NET version seems to be about 10-20% faster when benchmarked with 25 threads running for 30 seconds.)

Installation

Download and unzip the files into a local directory.

Make your first run using a "clientthreads=1" command like this:

C:\LocalDir>multithreadwebsimulator.exe -clientthreads=1 -configfile=test_publicservers.txt
-debugconsole=4 -maxwait=1 -minwait=1

When you understand how that works, increase "clientthreads" to 2, 3, 5, etc. and then start experimenting with your own local servers.

You MAY NEED TO set the following option in your "machine.config" file. (e.g., C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\CONFIG\machine.config) The default is "2"; I set mine to "64" and later "1024". (As soon as I changed this and saved on my original Windows 2000 Server development machine, I ran my application and my netstat -an showed lots and lots of connections to my remote machine (where before it showed only two). 

<connectionManagement>
 <add address="*" maxconnection="1024"/>
 </connectionManagement>

Usage

 D:\Code\DotNet\CS\MultiThreadWebSimulator\bin\Debug>multithreadwebsimulator -?
 14:53:23.656 3 MultiThreadWebSimulator 1.3.0.1 (MS.NET) started.
 2006-2009 by Jonathan Lampe (http://www.lampefamily.us/jonathan)
 Options:
 -clientthreads = Number of client threads to spawn (default=1)
 -runperiod = Maximum run time in seconds (default=30)
 -minwait = Maximum run time in seconds (default=3)
 -maxwait = Maximum run time in seconds (default=15)
 -stagger = Number of web sessions to kick off at once (default=3)
 -debugdisk = Debug level - disk (default=MoreDebug)
 -debugconsole = Debug level - console (default=AllDebug)
 -logappend = y/n (default=n)
 -configfile = [name of file] (default=test.txt)

Command-Line Options

  • ClientThreads - This is the total number of web browser client threads you want to simulate.
  • RunPeriod - This is the amount of time (in seconds) that the application will run before it shuts down all web session threads and exits.
  • MinWait - This is the minimum amount of time each web session will pause between web browser actions ("clicks" or uploads/downloads).
  • MaxWait - This is the maximum amount of time each web session will pause between web browser actions. This value is also used to figure out how to stagger the launch of web sessions.
  • Stagger - This controls how about many web sessions will be launched in each MaxWait period of time. For example, if Stagger=4 and MaxWait=16 then about 4 new web sessions will be launched every 16 seconds (or about 1 every 4 seconds, through this is certainly not guaranteed).

In general, it will take about (ClientThreads/Stagger) * MaxWait seconds to launch ALL web sessions. For example, if you configure ClientThreads=60, Stagger=5 and MaxWait=15, it will take (60/5)*15 = 180 seconds to launch all 60 web sessions. (The number of active web sessions currently active will be displayed at the "SomeDebug" level until all web sessions are active.) Similarly, if you configure a value of RunPeriod that is larger than (ClientThreads/Stagger) * MaxWait, not all web sessions will be launched.

  • DebugDisk - This is the "to disk" debug level. Generally, this should be equal or greater than DebugConsole. (JGL recommends "4".) Despite the listed default, this parameter currently only takes DEBUG_LEVEL_NUMBERS.
  • DebugConsole - This is the "to console" debug level. Generally, this should be equal or less than DebugDisk. (JGL recommends "5".) Despite the listed default, this parameter currently only takes DEBUG_LEVEL_NUMBERS. 

 DEBUG_LEVEL_NUMBERS
 1 = Errors Only
 2 = Warnings Too
 3 = Success Messages Too (like statistics)
 4 = Some Debug Messages Too (like URLs tried)
 5 = More Debug Messages Too (like cookie and sleep info)
 6 = All Debug Messages Too (everything else)

  • LogAppend - This controls whether the associated debug log will be a brand new file (LogAppend=n) or a continuation of an old file, if one exists (LogAppend=y).
  • ConfigFile - This is the name of the configuration file from which link macros should be read. If no name is specified, the application will look for a file named "test.txt" instead.

Options File

A configuration file containing parameters and "link macros" is required. By default, this file was named "test.txt", but you can specify other filenames with the "-configfile" command-line option. This file should be in the same folder from which the application is run.

The top section of the configuration file is for "name=value" parameters. The bottom section (everything after the "***LINKS***" line) is a "link macro". Any lines that begin with "#" or "'" are considered comments and are ignored.

Sample Configuration File

 # Please make sure macros are LOWERCASE
 # Valid macros in this section are:
 #  [threadid]
 UsernameMacro=tk__[threadid]
 PasswordMacro=a1s2d3
 #
 # Everything below the "***links***" line will be treated as part of the script
 # Valid macros in this section are:
 #  [threadid]
 #  [username]
 #  [password]
 #
 ***LINKS***
 # Sign on
 https://192.168.3.183/ Not Signed On
 https://192.168.3.183/theapptotest.php?transaction=signon&username=[username]&password=[password]  Signed onto
 # Click around some pages
 https://192.168.3.183/theapptotest.php?Arg12=folders Folders and Files
 https://192.168.3.183/theapptotest.php?Arg12=logview Current Filter
 # Sign off
 https://192.168.3.183/theapptotest.php?transaction=signoff Signed off successfully
 https://192.168.3.183/theapptotest.php?Arg12=infotech Return to the

"name=value" Parameters

  • UsernameMacro - This value will be consumed by the "[username]" macro in the link macros. Generally, this should be something like "tk__[threadid]" so that thread #0 signs on as "tk__0", thread #1 signs on as "tk__1", etc.
  • PasswordMacro - This value will be consumed by the "[password]" macro in the link macros. Generally this is a static value (such as "a1s2d3"). This could also be something like "passwd[threadid]" so that thread #0 signs on as "passwd0", thread #1 signs on as "passwd1", etc.

Link Macros

Link macros ALL take the form of: ACTION [Option] Options may contain spaces - no double or single quotes are needed. (Everything after the Action is assumed to be part of the option.)

There is currently just one type of supported link macro:

  • (LINK) [expected_response] - This will send a "GET" request to the URL in (LINK). If a value for [expected_response] is provided, the request will only be considered successful if a response that contains the expected response value is returned. Example: "https://192.168.3.183/human.aspx?Arg12=logview Current Filter"

Link Macro Macros

There are several macros that may be used inside (LINK) values.

  • [threadid] - The ID (e.g, "0", "1" or "123") of the current thread.
  • [username] - The parsed value of the UsernameMacro. For example, if the value of UsernameMacro is "x[threadid]", then Thread #2 would interpret "[username]" as "x2"
  • [password] - The parsed value of the PasswordMacro. For example, if the value of PasswordMacro is "y[threadid]", then Thread #4 would interpret "[password]" as "y4"

Hints

The red/yellow/white/grey console text toggles do not work on CMD shells with non-standard colors. (Also, the console text toggles may not work at all under Mono.)

To quickly simulate active web users that do something about every 10 seconds, JGL recommends the following command-line options with various values of ClientThreads: multithreadwebsimulator -clientthreads=60 -runperiod=300 -minwait=5 -maxwait=15 -debugdisk=5 -debugconsole=4 -stagger=5

To quickly eyeball if a machine can handle a few hundred sessions, JGL recommends the following command-line options: multithreadwebsimulator -clientthreads=300 -runperiod=1800 -minwait=5 -maxwait=15 -debugdisk=5 -debugconsole=4 -stagger=3 ...and suggests watching for the point in the log where errors and timeouts begin to show up. Then you can backtrack to a previous "X of Y web sessions are now running" message. The value of X that you find there is an eyeballed estimate of "how many active web users will this machine support"

You may want to turn IP lockouts OFF on the target web application before running tests. A flurry of hiccups can shut down ALL your tests if it is left on.

Example

D:\Code\DotNet\CS\MultiThreadWebSimulator\bin\Debug>multithreadwebsimulator -cli
entthreads=6 -runperiod=30 -minwait=3 -maxwait=5 -debugdisk=5 -debugconsole=4 -s
tagger=2
16:29:54.187 3 MultiThreadWebSimulator 1.1.0.2 (MS.NET) started.
16:29:54.187 6 Parsing command-line option #0 '-clientthreads=6'
16:29:54.187 6 Processing command-line option 'clientthreads=6'
...
16:29:54.187 6 Parsing command-line option #5 '-debugconsole=4'
16:29:54.187 6 Processing command-line option 'debugconsole=4'
16:29:54.187 3 Starting 6 client threads.
16:29:54.187 3 Each thread will pause 3 to 5 seconds between each request.
16:29:54.203 3 2 new web sessions will be started every 5 seconds.
16:29:56.265 4 1 of 6 web sessions are now running.
16:29:56.500 4 Cli#0 I#0(0) trying 'https://internal.targetmachine.us/'...
16:29:56.906 4 Cli#0 I#0(0) got response in 407ms.
16:29:56.906 4 Cli#0 I#0(0) found expected text 'Not Signed On' in response of l
ength 5798.
16:29:59.562 4 Cli#1 I#0(1) trying 'https://internal.targetmachine.us/'...
16:29:59.687 4 Cli#1 I#0(1) got response in 125ms.
16:29:59.687 4 Cli#1 I#0(1) found expected text 'Not Signed On' in response of l
ength 5801.
16:30:00.296 4 2 of 6 web sessions are now running.
16:30:01.531 4 Cli#0 I#1(2) trying 'https://internal.targetmachine.us/testedapp.
php?transaction=signon&username=tk__0&password=a1s2d3'...
16:30:02.062 4 Cli#0 I#1(2) got response in 531ms.
16:30:02.062 4 Cli#0 I#1(2) found expected text 'Signed onto' in response of len
gth 13170.
16:30:02.734 4 Cli#3 I#0(3) trying 'https://internal.targetmachine.us/'...
16:30:02.968 4 Cli#3 I#0(3) got response in 234ms.
16:30:02.968 4 Cli#3 I#0(3) found expected text 'Not Signed On' in response of l
ength 5798.
16:30:03.296 4 3 of 6 web sessions are now running.
...
16:30:11.296 4 6 of 6 web sessions are now running.
16:30:11.296 3 All 6 web sessions are now running.
...
16:30:23.312 4 Cli#3 I#4(27) got response in 609ms.
16:30:23.312 4 Cli#3 I#4(27) found expected text 'Standard Mailboxes' in respons
e of length 11800.
16:30:24.546 4 All client threads have been stopped.
16:30:24.546 3 * * * STATISTICS * * *
16:30:24.546 3 Cli#0: 6 of 6 OK (390ms)
16:30:24.546 3 Cli#1: 6 of 6 OK (395ms)
16:30:24.546 3 Cli#2: 4 of 4 OK (546ms)
16:30:24.546 3 Cli#3: 5 of 5 OK (521ms)
16:30:24.546 3 Cli#4: 4 of 4 OK (578ms)
16:30:24.546 3 Cli#5: 3 of 3 OK (421ms)
16:30:24.546 3 TOTAL: 28 of 28 OK (467ms) 100% OK
16:30:24.546 3 Average Transactions per Second: 0.933333333333333
16:30:24.546 3 Maximum (Peak) Transactions per Second: 3
16:30:24.546 3 MultiThreadWebSimulator stopped.

License

This software, and all accompanying files, data and materials, are distributed "AS IS" and with no warranties of any kind, whether express or implied.

History

In 2006 I wanted a multi-threaded, command-line web simulator application that could read up a list of commands to execute, apply a few macros to simulate different users and perform some custom MOVEit DMZ-specific upload and download actions. (These commands have been DISABLED in this public release.)

In 2008 Andy White rewrote this application for Ipswitch's use (that code remains Ipswitch property) and several other applications, including commercial applications, were used to take over its previous duties. In early 2009 I built a special, hopefully final version of this utility after removing some MOVEit DMZ-specific items and released it to the public.

Frequently Asked Questions (F.A.Q.)

Q: How do I report a bug?
A: Drop me an email (see contact information at bottom of page). However, I can't promise I'll fix your bug.

Q: Is support or commercial licensing of any kind available?
A: Sorry, but no.

Q: Do you plan on writing a more platform-independent or Linux version?
A: Natively? No. This application runs under Mono as described above, but I do not plan on reimplementing it again.

Q: Do you plan on writing a GUI or web-based version?
A: No.

Q: Is the source code available?
A: Maybe - ask nicely. (I'm still thinking about general release of this; I used a hack based on a deprecated interface to get this utility to quit complaining about certain self-signed SSL certs and I'm not particularly proud of it.)