Overhaul: Test Automation with SoapUI using JsonBuilder and JsonSlurper

I was looking into various ways to do more efficient creation of Json on the fly using the techniques I’ve described in my previous posts. However, while you can do it these ways (which would have to include writing your own parser) there’s a much easier solution, which I’m just glad to have realized sooner rather than later.

I still believe the best way to do automated regression is in a script library away from SoapUI; it gives far more flexibility when the need to make changes inevitably occurs. Keeping that in mind the following approach is not only easier but not hard to implement. Instead of passing each parameter per column in Excel, you can pass the entire Json string in one column. Then, using a combination of the JsonSlurper and the JsonBuilder/StreamingJsonBuilder, you can create valid Json.

So, how do you do it. Consider the following code below, which can be used for any Json you would pass in. The ‘rawJsonString’ is created because if someone pastes json into the Excel cell it can be in a ‘pretty’ format. At least this way we strip out any returns and newlines and let the inbuilt JsonSlurper handle the rest.

The simple function, jsonToMap, creates a new JsonSlurper, parses the text (just like you could do for your assertions), and returns the result.

Finally, we need to actually create our json that will appear in the body of our Rest request. Here’s where the StreamingJsonBuilder comes in. We use StreamingJsonBuilder because we aren’t manipulating or have a need at all for keeping anything in memory. After doing some research this is a cleaner solution and behaves similarly to the JsonBuilder referenced in earlier posts.

The function, CreateObject, takes the parameters ‘root’ and ‘map’. Root is taken from the createJSON function. In most calls you’re either going to have a root with a bunch of given parameters, lists, etc, under it, or it’s just going to be a list of parameters. Since you’re pasting in the rest of the Json in the value of your test case parameter you just need that one value before the output of jsonToMap. If you wanted to you could take out root entirely and just pass the whole json including the root. I think this way provides more possible flexibility going forward. The rest of the function should be self explanatory.

The code that would be in your SoapUI Groovy script is:

def req = new Json()
req.createJSON(testRunner,4,'RootObject')

The code that is used to create the json:

public class Json {

	def createJSON(testRunner,testStepLocation,root) {
		//properties
		def request = testRunner.testCase.getTestStepAt(testStepLocation).getTestRequest()
		def tcProps = testRunner.testCase.getProperties()
		def map
		def rawJsonString
		def prettyPrintJson
		
		//grab objects
		for (p in tcProps) {			
			if (p.value.getValue() != "") {	
					prettyPrintJson = p.value.getValue()	
					rawJsonString = makeSingleLineJson(prettyPrintJson)
					map = jsonToMap(rawJsonString)
			}
		}			
		if (root != null) {
			def object = CreateObject(root, map)
			request.setRequestContent(object)
		} else {
			request.setRequestContent(rawJsonString)	
		}
	}
	
	//create Object
	def CreateObject(root, map) {
		def jsonWriter = new StringWriter()
		def builder = new groovy.json.StreamingJsonBuilder(jsonWriter)				
		def r = builder {						
				"${root}"(map)						
			}				
		def json = groovy.json.JsonOutput.prettyPrint(jsonWriter.toString())
		json = groovy.json.StringEscapeUtils.unescapeJava(json)
		return json
	}
	
	//convert the given json
	def jsonToMap(someJson) {
		def slurper = new groovy.json.JsonSlurper()
		def result = slurper.parseText(someJson)
		return result	
	}
	
	def makeSingleLineJson(prettyPrintJson) {
		def rawJson = prettyPrintJson.replaceAll("\\r?\\n","")
		return rawJson	
	}
}

SoapUI Groovy: Splitting out your scripts

Please see updated post here for a simpler solution to passing the data in. I still use the script library for holding all my scripts.

In previous examples I used if/else statements to separate my test case property data. I also used the same scripts over and over to parse arrays, etc. As JSON calls become more complex you need a new way to create the JSON on the fly while keeping everything legible. To accomplish this I combined the method described in my post about creating objects (here) with a switch statement and some reworking of the overall code.

Lets say we have a JSON call that needs to look like this:

{
    "Object1" : 
    {
        "Object2" : [
            {    
                "Something" : 1,                
                "Something2" : "string",                
                "Something3" : ["strings"]
            }],        
        "Object3" : [
            {
                "Something" : "string",                
                "Something2" : 1,                
                "Something3" : true
            }],        
        "Something" : "string",        
        "Something2" : 1,        
        "Something3" : true      
    }
}

As passed through Excel spreadsheets with 3 columns (Object1, Object2, and Object3), my test case properties look like this:

objectProperties

My ‘main’ groovy script which is called from SoapUI to create the JSON:

package qa

public class CreateObject {

	def createJSON(testRunner, testStepLocation) {

		//properties
		def request = testRunner.testCase.getTestStepAt(testStepLocation).getTestRequest()
		def tcProps = testRunner.testCase.getProperties()
		def object1 
		def object1Map = [:]
		def object2
		def object2Map = [:]
		def object3 
		def object3Map = [:]
				
		//grab objects
		for (p in tcProps) {		
			if (p.value.getValue() != "") {		
				switch(p.key) {
			 
				case 'Object1' :
					object1 = p.value.getValue()
					object1Map = new Tools().fixObjectToMap(object1)	
					break

				case 'Object2' :
					object2 = p.value.getValue()
					object2Map = new Tools().fixObjectToMap(object2)	
					break
			
				case 'Object3' :
					object3 = p.value.getValue()
					object3Map =  new Tools().fixObjectToMap(object3)	
					break
				
				default: 'Default'			
				}
			}
		}	
		//set request json
		def c = new Object().Create(object1Map,object2Map,object3Map)		
		request.setRequestContent(c)		
	}			
}

Some of the ‘Object’ script:

package qa

public class Object {

	//create 
	def Create(object1Map,object2Map,object3Map) {

		def builder = new groovy.json.JsonBuilder()				
		def r = builder.Object1 {		
			if (object2Map.size() != 0) {
				Something3([{
					object2Map.each() {
								key,
								value ->
									if (key.equals('Something3')) {
										value = new Tools().simplisticParse(value, String)
										"${key}" value
									} else {
										"${key}" "${value}"
									}
								}	
				}])
			}
			if (object3Map.size() != 0 ) {
					Object3([object3Map]) 
			}
			if (object1Map.size() != 0) {
					Object1([object1Map])
			}				
		}		
		def json = builder.toPrettyString()
		json = groovy.json.StringEscapeUtils.unescapeJava(json)
		return json
	}	

The ‘Tools’ script referenced above:

package qa

public class Tools {

	//parser for converting strings to arrays
	def simplisticParse(String input, Class requiredType) {
		input.dropWhile {
			it != '['
		}
		.drop(1)
		.takeWhile {
			it != ']'
		}
		.split(',') *.asType(requiredType)
	}
	
	//take what was given and take out the tabs and then make a map from it
	def fixObjectToMap(original) {			
		original = original.replaceAll("\\r?\\n","")			
		def result = original.split('&').inject([:]) { map, token ->
		token.split('=').with { map[it[0].trim()] = it[1] }
		map
		}
		return result
	}	
}

Important things to note:
– All of these scripts reside in the same directory so can be referenced in the form of ‘X_file().Y_method’
– The ‘Object’ script actually contains all the different methods for creating JSON so that’s just a small section of what your overall script could look like

SoapUI Groovy: Using JSONBuilder to create JSON from TestCase properties that contain an Object

Please see updated post here for a simpler solution.

I looked around and couldn’t find a simple solution to creating JSON from test case properties while one of the properties already had a written out object. My requests ended up having quotes where they shouldn’t have, badly formed JSON, or just blank in some areas.

Let’s say this is what my SoapUI TestCase Properties used to look like when grabbed from Excel:

tcPropsExample

So, the JSON is more or less written out for my MyObject property. I also tried different combinations, for example, writing out “MyObject” : {“Date”:…..etc} or { “MyObject” : {“Date” :……etc} and using that as the property value, but nothing worked out of the box like I wanted it to.

Now, I still have other properties in there that I want to make part of my JSON call, in the end I want a structure like:

{
    "A": "value",
    "DefinedObject1": {
        "X": "value",
        "Y": "value"
    },
    "DefinedObject2": {
        "Date": "value",
        "Param1": "value",
        "Param2": "value"
    }
}

I don’t know if there’s a better solution to this, but instead of passing in well formed JSON as a value I write out the JSON using ‘=’ instead of ‘:’ and ‘&’ instead of ‘,’.

Which ends up looking like this in an editor:

json1

Then I use this Groovy script for creating my JSON body:

package qa

public class MyClass {	

	def createJSON(testRunner,testStepLocation) {

		//properties
		def request = testRunner.testCase.getTestStepAt(testStepLocation).getTestRequest()
		def tcProps = testRunner.testCase.getProperties()
		def map1 = [ : ]
		def map2 = [ : ]
		def object1
		
			//grab all the properties from the test case and put them in a map as long as they aren't empty
			for (p in tcProps) {
				if (p.value.getValue() != "") {
					if (p.key.equals('A')) {
						map1.put(p.key, p.value.getValue())
					} else if (p.key.equals('X') || p.key.equals('Y')) {
						map2.put(p.key, p.value.getValue())
					} else if (p.key.equals('MyObject')){
						object1 = p.value.getValue()
					}
				}
			}

			if (object1 != null && object1 != "") {	

			//remove line break characters from the string
			object1 = object1.replaceAll("\\r?\\n","")
			
			//split the string for the object into a new map
			def object1Map = object1.split('&').inject([:]) { map, token ->
				token.split('=').with { map[it[0]] = it[1] }
				map
			        }
			}			
			def builder = new groovy.json.JsonBuilder()

			if (object1 != null && object1 != "") {	
			  def root = builder {
				map1.each() {
					key,
					value -> "${key}" "${value}"
				}
				DefinedObject1 {
					map2.each() {
						key,
						value-> "${key}" "${value}"
					}					
				}	
				DefinedObject2 {		
					object1Map.each() {
						key,
						value -> "${key}" "${value}"
					}
				}
			  }
			} else {  //could add more if/else statements here to cover all scenarios but you get the idea
			  def root = builder {
				map1.each() {
					key,
					value -> "${key}" "${value}"
				}
				DefinedObject1 {
					map2.each() {
						key,
						value-> "${key}" "${value}"
						}					
					}	
				}
			}			
		def json = builder.toPrettyString()
		json = groovy.json.StringEscapeUtils.unescapeJava(json)
		request.setRequestContent(json)
	}
}

I think the comments inline make it pretty self-explanatory but just to clarify this is what happens:

1) Grab all the TestCase properties and split them up into different maps
2) If it’s MyObject as the key (property name) then instead just grab the value and assign it to a string variable
3) Strip said string of any new line characters if not null/empty
4) Fancy code that I got from StackOverflow here that basically just says “Split up my string based on X and Y delimiters and make it a map”
5) If statements to cover different structure if variables are null/empty
6) Create the JSON based on those three maps we just created

Enjoy!

Making things organized…

I recently compared a few other tools to SoapUI, but for the price and compared to what has already been done, nothing stuck out.

Here’s a quick recap with some improvements that were necessary..

Say you have a group of testers all using the same SoapUI composite project. JSON calls are normally a mix of POSTs and GETs. For the POSTs, you can setup a WSDL-like environment that are separated by categories and have various calls under them.

For POSTs regression you can have a loop that’s the same for each API (more complex ones can have their own scenario based calls):

– Datasouce – Excel
– Property Transfer
– CreateValidJSON – Groovy Script
– Add Assertions – Groovy Script
– JSON Request
– ExcelDataSink – Output Pass/Fail Results to Excel Docs
– OutputRequestResponseFile – Groovy Script
– Loop

You can create other test cases or suites to cover specific testers if you want. You can then create your own cases to test your enhancements or bugs and check into SVN (if you use a repository) against the given Jira number. Handy plugin. Regression cases can be tracked by entering them into the Excel Datasources and again checked in against the given Jira number. To get around multiple testers updating regression sheets at once you can put locks on all your Excel documents.

Getting back to SoapUI…I really, really wish it had a Composite Action -> Test Suite option. This would allow the ability to save a test suite/reload a test suite without having to do a full project save/update. I saw it as a future enhancement so we’ll see if it’s ever implemented. For now, you need to refresh the whole project at the end of a cycle or when you need to see each others updates. Not a huge deal.

Recent improvements to SoapUI…

In version 5.0, they made a lot of headway (which has been nice) from when we first stated setting up REST testing. For one, they’ve upgraded the Groovy jar file that’s included (yay!). They’ve also made the creation of requests based off URI more user friendly and updated the parameters GUI so it’s less..for lack of a better word…buggy. I’m pretty sure they’ve fixed either a memory leak bug or an issue related to it, because even without updating the JVM memory parameters I’m no longer occasionally crashing.

Things I’m working on…

So, going forward I’ll be looking at better ways to pass entire objects through from Excel to SoapUI to then convert, using Groovy, into valid the JSON. That’s been a little problematic so far. I’ve done it, but it’s not pretty and is really a trick of the whole system that I can’t expect other testers to deal with.

SoapUI: Output TestCase Results to Excel

As you can tell from the rest of my posts, this is again using SoapUI Pro and making use of the DataSink test step and the Extended Script Library.

So, in my test case results, I’d like to see the test case number, description, status, assertion, and the date it was run. In my Excel docs I have each of those as a heading and then SoapUI just starts at the next row. Since I take my initial test cases also from Excel I get the test case Id and description from that Datasource.

The DataSink step I set to be Excel (from the dropdown) and then I get the option to add properties on the left. As mentioned, I create five properties:
1) TestCaseId
2) TestCaseDescription
3) Status
4) Assertions
5) Date

TestCaseId and TestCaseDescription are easy since they are already in my Datasource:

${Datasource#TCId}
${Datasource#TCDesc}

Status can just be called using a Groovy script inline, like so:

${=testRunner.results[testRunner.results.size()-1].status}

The ‘1’ in the above example is the location of the test request in relation to the DataSink.

Assertions references a script in my Script Library, which could look like:

${=new qa.GetFailedAssertions().outputResult(testRunner)}

And then the actual script looks like:

package qa

public class GetFailedAssertions {

	def outputResult(testRunner) {	
		def tcName = testRunner.testCase.name
		def assertionList = testRunner.getTestCase().getTestStepByName(tcName).getAssertionList()
		
		for( assertion in assertionList ) 	{
			for(e in assertion.errors ) {
			return "Assertion Failure: [" + e.message + "]"
			}
		}
	}
}

Date is just today’s date with the time, again inline Groovy script:

${=new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm").format(new Date())}

Now when I run my case all these results are put into the files referenced in the File parameters of the DataSink.

SoapUI: Composite projects and SVN

If you have multiple testers that need to use the same project you can make use of the composite project option in SoapUI Pro and some type of SVN.

So, first you want to have some kind of SVN repository setup where you are going to sync your project to. Once you have this created you can create your folder structure under it that you want to use with SoapUI. For example, if you use Groovy scripts for automation you can also include those in a folder and commit them to the SVN so everyone can also have access to update them.  For this example we’ll say your folder structure looks like this under a root folder called ‘SVN’:

folderstructure

Now you want to open SoapUI.

Save your project under the Projects directory we just created. Select it at the root level in SoapUI. You’ll see project properties at the bottom:

soapuiProjProps

You want to make sure Composite project is set to true. Now instead of your projects being a single xml file they’re broken up so they look something like this:

testSuites

Each test suite is broken up into its own folder. There are also element.order and settings.xml files. If you look at these files (in notepad or some variety of) then you can see that element.order is just the order of whatever items are in the level you’re looking at. At the root it’s the order of the test suites, inside a test suite it’s the order of the cases. Settings.xml is the various environment and project properties at the root level and the test suite ones at that level.

Remember the Scripts folder?  Well you can set your global SoapUI properties to be the folder that you plan on putting on SVN.

ScriptsSettings

Now that everything is setup you can go to the root level (C:\SVN) and do your SVN Checkout. Then you’ll want to do the SVN Commit.

Everyone else using the project can now setup their own SVN folder and checkout your work. Note that in order to see new test suites/cases in SoapUI after an SVN update you need to do a Refresh Composite Project:

refeshproj

This will update your SoapUI to what is now in your local copy from SVN (which by doing the SVN update should now match what’s on the SVN).

Good rule of thumb, as if you were developing, would be to update at least once a day if not twice and Commit anything that is completed or you don’t want lost.  Typical scenario would be:

Tester A creates new test suite
Tester A saves SoapUI project (this is now saved to his/her local copy ONLY)
Tester A commits said project to SVN (if it’s a new test case in an already existing folder you can always just commit the folder)
Tester B does an SVN Update
Tester B goes to SoapUI project and does a Refresh Composite Project
Tester B sees Tester A’s test suite

SoapUI Groovy: Disabling a test step within a test case

If you want to disable a specific test step that’s in all your test cases, at the Setup Script level for your TestSuite, here’s what you can do:

def totalTestCases = testSuite.getTestCaseCount();

for(n in (0..totalTestCases-1))	{    
 	if (testSuite.getTestCaseAt(n).getTestStepByName("MyTestStep")){
 			testSuite.getTestCaseAt(n).getTestStepByName("MyTestStep").setDisabled(true)
	 	}
}

Enjoy!