How to Test

Parent Previous Next

How to Test.

The test bench can work with multiple test scenarios, each handling different input data, generating different results. Each scenario is managed as a test case.


Test Case Management.

You manage the test cases at the top of the Test tab-page:



You use the combo-box to switch among the created test cases and the drop-down button for managing them.


The drop-down menu contains these functions:


Function

Description


Add New

Add Named

Add Example

These functions are used to create and insert new test cases. When invoked, the created test case becomes the currently selected. Each test case must have a unique name and is presented by the combo-box in alphabetical order.

The Add New function uses a "New TestN" fast-naming scheme assigning consecutive numbers. They can be renamed later. The Add Named function allows for specifying a real name.

TIP: Try using the Add Example function to see how to work with cell values.

Copy New

Copy Named

These functions copies an entire test case to a new test case. The values specified for the execution flow parameters are included.

The Copy New function uses the "New TestN" naming scheme. The Copy Named function allows for specifying a real name.

Clear

Resets the execution flow parameters to their default values.

Delete Current

Delete All

These functions deletes either the current test case or all the test cases. If all test cases are deleted, a new empty test case is inserted.

Rename

This function allows for changing the name of a test case. Please note it might cause the test case gets a new position within the combo-box presentation order.



Execution of Step 1 - 2 - 3.

Each test case contains execution flow parameters - defined by a 3 step process that revolves around the execution of a VM like this:


When the test case is to be executed by the test bench, each step of the process is executed in consecutive order as described next.


Step 1. Set Cell Values.

Before calling VMExecute you have the option of assigning input values to the VM Cells, in which case step 1 must be selected. It is not mandatory and can be skipped.


The input values must be written in a certain way in order to be inserted correctly as typed values into the cells. An example of this is available by invoking the Add Example function as shown here:



As exemplified you may define one cell per line and use a positive or negative index value for the row and column. Please pay close attention how to write cell values for integers, blobs and strings.


Step 2. Main Function Arguments.

This step is mandatory and is always executed by the test bench. It is here we call the VM to let it execute the bytecode.



As defined by the Main Entry Point the Main function of the Main class requires the integer arguments x, y, z.


Step 3. Scan Cell Value Results.

Once the VM has finished executing the bytecode, the test bench provides the option to scan the cells and list any values found as execution results. The cells to scan are defined by searching a range of rows combined with a range of columns:



This exemplifies a scan that for each of the 7 rows in [0,2,4,5,6,7,9] search the cells of each of the 201 columns in [-100..100].


Execution Results.

When you click the button "Start Execution of Step 1 - 2 - 3" the following happens:


This example demonstrates how the result box presents the reported results:


Execution started at 20151219 16:00:24.106

 Using:

 - DLL file: Using C:\DATA\SCRAMBLECODE\vm32.dll

 - ByteCode: Using latest compilation in memory from 20151219 15:58:43.891


 VM creation time = 0ms.


 VM execution:

 - Execute Main(0, 0, 0)

 - Main returns 0

 Execution time = 0ms.


 Get cell values:

   0, 0, "This is a returned result."

   0, 1, 0xFFAA88

Execution stopped at 20151219 16:00:24.153


Note: The measured time is rounded to milliseconds - e.g. 0ms can mean 0.49ms. As demonstrated here between the start and stop time is some 47ms which seem unaccounted for, but this time gap relates primarily to loading/unloading the DLL, creating a scan of the cells and presenting the results.


TIP: For more time consuming processes it can be interesting to do the execution several times to observe variations in the measured time durations. Please note that bytecode doing file operations may experience larger variations due to the hard drive and/or malware software, than bytecode performing in-memory operations only.


Result Limitations.

When presenting cell values as results only the first approx. 1000 characters of each value are presented in the result box. This is a limitation of the result box. It is not a limitation of the cells. If you need to verify results with really big strings or blobs, they can easily be written to a file instead.


Using CallBack.

The test bench provides the VM with a callback function which it can use during the bytecode execution to call a function inside the host application (currently the test bench). This callback function simply writes the callback arguments to the result box and return zero to the VM, which then continues executing the bytecode.