Guides for SE student projects »

Automating the testing of text UIs

I/O redirection technique can be used to semi-automate the testing of text UIs.

Sections below explain how to use that technique in a small project.

Setting up

Given below re steps to set it up for a project.

  1. Create a folder to hold the relevant files e.g., [project root]\text-ui-test

  2. Add a runtest.bat (if you are on Windows) or runtest.sh (if you are on a *nix OS) into the folder, containing the script below.

    @ECHO OFF
    
    REM create bin directory if it doesn't exist
    if not exist ..\bin mkdir ..\bin
    
    REM delete output from previous run
    del ACTUAL.TXT
    
    REM compile the code into the bin folder
    javac  -cp ..\src\main\java -Xlint:none -d ..\bin ..\src\main\java\*.java
    IF ERRORLEVEL 1 (
        echo ********** BUILD FAILURE **********
        exit /b 1
    )
    REM no error here, errorlevel == 0
    
    REM run the program, feed commands from input.txt file and redirect the output to the ACTUAL.TXT
    java -classpath ..\bin Duke < input.txt > ACTUAL.TXT
    
    REM compare the output to the expected output
    FC ACTUAL.TXT EXPECTED.TXT
    

    #!/usr/bin/env bash
    
    # create bin directory if it doesn't exist
    if [ ! -d "../bin" ]
    then
        mkdir ../bin
    fi
    
    # delete output from previous run
    if [ -e "./ACTUAL.TXT" ]
    then
        rm ACTUAL.TXT
    fi
    
    # compile the code into the bin folder, terminates if error occurred
    if ! javac -cp ../src/main/java -Xlint:none -d ../bin ../src/main/java/*.java
    then
        echo "********** BUILD FAILURE **********"
        exit 1
    fi
    
    # run the program, feed commands from input.txt file and redirect the output to the ACTUAL.TXT
    java -classpath ../bin Duke < input.txt > ACTUAL.TXT
    
    # compare the output to the expected output
    diff ACTUAL.TXT EXPECTED.TXT
    if [ $? -eq 0 ]
    then
        echo "Test result: PASSED"
        exit 0
    else
        echo "Test result: FAILED"
        exit 1
    fi
    


  3. Update the javac and java commands in the script to match the name/location of your main class.
    If you are using packages, the two commands need to take the packages into account too.

  4. Add an EXPECTED.txt to the same folder, containing the expected output.

  5. Add an input.txt containing the input commands.

  6. Run the .bat/.sh file to execute the test.

    • If the actual output matches the EXPECTED.TXT, the test passes.
    • If the actual output differs from the EXPECTED.TXT, the script will report a failure.

Updating tests as the program evolves

The purpose of testing as explained in the previous section is to confirm there are no i.e., there are no unintentional behavior changesregressions. However, we often update the behavior of the program intentionally e.g., enhance an existing feature. Let's look at how to update our test set up in those cases.

Option 1: This is the ideal but more tedious approach.

  1. Determine how the expected behavior should change due to your update to the code.
  2. Update the EXPECTED.TXT file accordingly.
  3. Run the test to confirm the actual behavior is same as the updated expected behavior.

Option 2: This is a more practical shortcut.

  1. Run the test as per normal after updating the code. The test will fail because the new program behavior is different from the one given in the EXPECTED.TXT.
  2. Use a diff tool to compare the ACTUAL.TXT against the EXPECTED.TXT.
    Intellij IDEA can compare two files.
  3. Confirm the differences indicate the behavior has changed as you intended. If the differences are not as intended, your code is buggy; fix the code and repeat from step 1.
  4. Copy over the content of the ACTUAL.TXT to EXPECTED.TXT i.e., we accept that the actual behavior should be the new expected behavior. Rerun the test to confirm that it passes this time.

Troubleshooting

  • Problem: The ACTUAL.TXT and EXPECTED.TXT looks exactly the same but the test fails.
    Explanation: The likely cause that the line endings are different (not visible to the naked eye) because the two files were created in two different operating systems.
    Solution: You can use the dos2unix utility (available in git-bash and *nix operating systems) to convert a file to Unix format.

Authors:

  • Initial Version: based on se-edu/addressbook-level2, adapted by Jeffry Lum