Contents:
Simply put, Continuous Testing helps you get a better grade in 6.170, if you're already using Eclipse. In a controlled experiment in Fall 2003, students who used continuous testing were 50% more likely to turn in completely correct problem sets than those who only used continuous compilation (the standard Eclipse functionality), while spending no more time on their problem sets.
This tutorial assumes that you have completed Problem Set 0 using Eclipse. (If you've successfully completed the assignment, you may want to turn it in before continuing.) It also assumes that you have already installed continuous testing. If you are on an Athena machine, using the provided runeclipse script, continuous testing has already been installed for you. If you are on a personal workstation, please follow the on-line instructions to install the continuous testing feature into Eclipse. You will need to have installed Eclipse version 3.2M4. If you upgrade from a previous version, you will need to reconfigure your default workspace, but all other settings should carry over.
With continuous testing enabled, as you edit your code, Eclipse runs your tests quietly in the background, and notifies you if any of them fail or cause errors. Continuous testing builds on features of Eclipse with which you are already familiar, such as compile error notification: As soon as you write code that contains a compilation error, Eclipse highlights the error in the text and the margin, and creates a task in the Problems table. With continuous testing, you get the same kind of notification when you write or edit code that causes one of your tests to fail.
The developers of continuous testing are very interested in your feedback. Please contact us with any and all comments or complaints. You can also ask a staff member, such as an LA, for help
Since
continuous
testing is already installed, it's time to enable
it for our project. First, we need to make sure that continuous testing's
feedback will not be filtered out. In the Problems view, click the "filters"
button on the upper
right: it has three arrows pointing to the right. A dialog box will
appear. Look for the checkboxes labelled "Show items of type:". Make
sure that Test Failure is checked.
Now, we will enable continuous testing on the psets project:
All of your tests will run in the background immediately after you enable continuous testing. If you had closed or hidden the JUnit view in the past, it may have been brought to the front. If you wish, you can prevent this behavior by clicking a white downward arrow in the upper right of the JUnit view, and selecting "Active on Error/Failure only". You may notice multiple failure markers in the Problems View, if you still have failing tests.
Let us take a look at a Test Failure. If you've already successfully completed problem set 0, you should now introduce a bug into your code. For example, change HolaWorld.getGreeting to read:
public String getGreeting() {
return "No hablo Espanol";
}
In the Problems View, you will notice a Test Failure marker for testGreeting(ps0.HolaWorldTest). The method testGreeting() is currently as follows:
public void testGreeting() {
HolaWorld world = new HolaWorld();
assertEquals(HolaWorld.spanishGreeting, world.getGreeting());
}
We expect this
test to fail since the two strings being compared are not equal.
You'll see several indications of the failing test, similar to what you see when there
is a compile error in your code:
in the
margin.
Although the same kind of notifications are used for compile errors and test failures, the two kinds of problems are different. Most importantly, compile errors are usually indicated near the code that needs to be changed, whereas test failures are indicated on the test method that failed. To fix a test failure will often require making a change in the code being tested, somewhere far away from the failure notification.
Before we go farther, it's worth quickly pointing out a few things that you might expect to see, but don't. Continuous testing tries hard not to bother you unless there's something you really need to know.
First, notice that the tests being run by continuous testing do not appear in the the JUnit result view. This place is reserved for launches that you initiate manually. If you would like to see the results of a test run by continuous testing, select the menu "Run > Run History", and select the name of your continuous testing launch configuration from the list. Or, if there is a test failure marker in the Problems view, you can right-click it, and select "Rerun" (see "Rerun Tests" below.)
There are other subtle ways that continuous testing tries to stay out of your way. First, if there is a compile error anywhere in the project you are editing or the project containing the tests, or in those projects' dependencies, no tests are run. Second, even if two projects are associated with the same test, only one failure marker per test is ever created.
Right click on the test failure that has currently been created in the
Problems View. You'll see some standard options, including "Go To",
"Copy", and "Properties", as well as four options specific to
continuous testing:
Try selecting Rerun now. Notice
that the "T" icon at the left of the Problems View briefly changes from
red () to gray (
), then back to red. The gray means that
continuous testing is currently planning to
rerun the indicated test, and doesn't yet know the result. If the
rerun test fails, the icon is switched back to red. If the rerun
test passes, the problem is deleted.
to show the stack trace of the failed assertion or exception that
caused the selected test to fail. You can double-click
on elements of the stack trace to visit the source code location
indicated. Users often drag the JUnit Failure
Trace view to its own location just to the left of the Problems View, so it
can be easily referenced.
public String getGreeting() {
return "Hola Mundo!";
}
This may not be the correct solution to the problem in PS0, but it is a quick way
to see how test failure markers are added and removed from the Problems View.
Upon saving this change, the tests will rerun, and the Test Failure marker
for testGreeting(ps0.HolaWorldTest) will be removed from the Problems View.
Congratulations on completing the tutorial. Again, the developers of continuous testing are very interested in your feedback and questions. Please contact us with any and all comments or complaints. Also feel free to use the LAs and other course staff.
If you get an error message like "plug-in edu.mit.lcs.pag.continuoustester was unable to load class edu.mit.lcs.pag.continuoustester.project.ContinuousPropertyPage", that indicates that Eclipse was unable to load a plug-in's classes. To get a detailed error message that's more likely to pinpoint the root cause of these problems, users should look in the error log (.metadata/.log in the file system, or the Error Log view in the UI) for a stack trace near the time of startup complaining about the continuoustester plug-in.
When there is a classloading problem, the two most common problems are: