Validation

MITSIM is tested and calibrated on a number of networks with varying structure and complexities. Aided by animation graphics of vehicle movements, unrealistic behavior caused by limitations in models and errors in parameter values as well as implementation mistakes were detected and corrected. In this section, we describe a validation study conducted using a data set provided by UC Berkeley Path Program's Freeway Service Patrol Project. This data set was acquired from 16 detector stations on a 5.9-mile stretch of I-880 around Hayward, California. The network contains 4 on-ramps and 6 off-ramps (see Figure 3-14). The left lane is a HOV lane. The traffic counts, speeds, and occupancies aggregated over 5-minute time intervals are used in this study.

  
Figure 3-14: I-880 north freeway network

Using the observed traffic counts and speeds during a 4-hour time period for a number of days, time dependent O-D matrices were first estimated using the method of [Ashok and Ben-Akiva(1993)]. The average departure rate was 9,430 vehicles per hour (vph). of these vehicles were classified as HOV, buses, and trucks. Traffic counts, speeds, and occupancy for the period from 6:50 to 9:10am were collected at each detector station for 5-minute intervals and averaged over the 5 simulation runs. The first 10 minutes (i.e. two time intervals) are treated as ``warm-up'' periods and excluded from the data collection.

Figures 3-15(a)-(c) show scatter plots of the simulated and actual data. The points in these scatter plots indicate that the simulated traffic counts fit the actual data reasonably well. Simulated speeds and occupancies exhibit poor fit in some cases. The contour plots in Figure 3-16 show the evolution of speed and occupancy over time and space. A closer look into these data reveals that most of the outliers in Figures 3-15 (b) and (c) occur at two particular sections and may be caused by the following:

  
Figure 3-16: Contour plots of field and simulated speeds and occupancies

• Section between Tennyson ramps: For detector stations 17 and 4 located in this area there is a significant over-estimation of the speeds and under-estimation of the occupancies between 8:00 and 8:40 am. The variance of the simulation output over the 5 runs is also high during this time period. The animation graphics show that the weaving traffic in this section is heavy and the queues upstream of the merging area occasionally back up to the detector stations. As a result the sensor output is inaccurate.
• Section between A-Street On-ramp and SR238 Off-ramp: Five detector stations (20, 7, 1, 3 and 16) are located in this area. In the first hour of the simulation, there is significant over-estimation of speeds and under-estimation of occupancies at these stations. The differences between simulated and field data are more persistent at the downstream end. This is probably due to congestion occurring at the downstream section outside the network boundary, which is not taken into account in the simulation. Indeed, there are 3 more ramps located within 1 mile downstream of detector station 16 and the freeway changes from 5-lanes to 3-lanes. Unfortunately, data in this area is not available for simulating the operations and incorporating their impact.

  
Figure 3-17: Relationship between speed and occupancy in field data

The overall performance of the simulation can also be evaluated using the statistics listed in Table 3.1. These statistics are based on [Pindyck and Rubinfeld(1991)] and documented in Appendix E. Detector station 16 is excluded in the calculation of these statistics.

Table: Simulation errors in the I-880N network

Performance Measure		Flow	Speed	Occupancy
RMS error		30.99	8.82	2.80
Mean error		-3.94	3.21	0.39
Mean percent error (%)		-0.62	9.83	7.81
RMS percent error (%)		6.45	29.36	21.13
Correlation Coefficient		0.92	0.34	0.51
Theil's inequality coefficient		0.0303	0.0770	0.1324
325mmProportions of inequality		0.0162	0.1329	0.0192
		0.0016	0.4017	0.1117
		0.9822	0.4655	0.8691
Number of data points		390

Flow is the number of vehicles passing a detector station in 5-minute intervals;

Speed is the harmonic mean speed in mph;

Occupancy is measured in percentage of time that a detector is activated by passing or stopped vehicles.

The above results were obtained using the default values for various parameters in the simulation model. The only calibration attempt was made with respect to several parameters in the car-following model. The parameters , and of the car-following model in Eq (3.11) are based on [Subramanian(1996)], who estimated these parameter values using disaggregated data on vehicle trajectory collected from a freeway section of I-10 near Washington D.C. [Smith(1985)]. These values are , , , and , , (distance is measured in meters, speed in m/sec, and acceleration in m/sec ). The and are set to 0.5 and 1.36 seconds respectively. The step size for advancing vehicles is set to 0.2 seconds.

The lane-changing model used in MITSIM is currently undergoing extensive calibration and validation using detailed data on driver behavior and traffic conditions on various facilities [Ahmed et al.(1996)].