About the Author: Randy Iwasaki is the former director of the Contra Costa Transportation Authority and also served as the Director of the California Department of Transportation. He currently works with Amazon Web Services, a CALCOG partner.
Delivering a regional transportation plan (RTP) with a sustainable communities strategy (SCS) is a big task. That is particularly true for the Southern California Association of Governments (SCAG), which must chart the mobility of a region encompassing a population of more than 19 million people in over 38,000 square miles.
A core component of the RTP/SCS is a data-hungry Activity-Based travel demand Model (ABM). This tool forecasts the daily activities and travel patterns based on socio economic characteristics of more than 10 million households and accounts for the effect of new transportation and land development investments. In addition, the model generates performance indicators, air quality conformity analysis, and progress on environmental justice issues. Soon, it will also be able to be used to analyze the impact of infrastructure investment, pricing policies, and improvements in alternatives like active transportation.
But all that takes data–a lot of data. It can take three days to generate a model run for a single scenario. The typical output for such a run is three 300 gigabytes of data. And process for the current Connect So Cal plan analyzed at least 28 scenarios. Again, that is a lot of data. And with SB 743 implementation, the need keeps growing.
A Silver Lining
So where can an MPO house all of this data? The traditional approach was to buy a couple of servers and let the modelers figure it out. But the quantity of data—and the risk of having all the data in one location—is driving more public agencies to cloud based services.
SCAG chose a different route. They migrated their on-premises data storage and analytical tools to the Amazon Web Services (AWS) Cloud. They chose the cloud because of the massive amount of data that need to be produced and stored for more than 10 years. But the compute power and reliability were also key factors in their decision. With the AWS cloud, SCAG no longer had to calculate how much storage they need or how much computing power to buy since capacity can be scaled up or down, on-demand.
According to SCAG’s modeling & forecasting manager, Dr. Hsi-Hwa Hu, “We’re not running the model every day. We may need to run 5-10 models at the same time, and this is a constraint for physical servers. The cloud gives us flexibility.”
The Amazon Elastic Compute Cloud (Amazon EC2) service has more availability than traditional servers. The service not only provided SCAG with more availability but allowed them to complete the model runs faster. And if they need more, they can spin up additional model runs (called “instances” in cloud speak) within minutes.
“Speeding up model output time and providing storage elasticity was our goal in starting to explore cloud options. We have very tight turnaround times. We could not afford to have the model crash in the middle of a run,” said Julie Stoyer, CIO of SCAG. The elasticity of the cloud allows SCAG to be able to meet the highest peak demands but not have those investments sit idle when they are not needed.
Using a master services agreement with AWS, the procurement process was very quick. “Once we migrated to the cloud, we were able to free up IT resources to work on other much needed services,” said Shroyer.
In the end, of course, its about providing better service to the communities of Southern California. For SCAG, the ultimate benefit is a better performing transportation model, which leads to better decision making around transportation investments and achieving other important goals relating to equity, climate, and air quality.
More Clouds Forecasted
Looking forward, SCAG is developing a cloud-hosted regional data platform in partnership with Esri, a geographic information system company. There are 191 jurisdictions in the SCAG region, and these jurisdictions’ data are inputs into the model. Now, through an AWS hosted portal, local agencies can upload their data directly into the regional data platform. This kind of direct interaction is a big time saver, and it reduces data entry error. The data is then available to each community as they need them (like for a general plan update).
According to Julie, the success of these projects is the close partnership between the IT and Modeling Group at SCAG. “We created a collaborative environment, the teams work together to deliver the state of the art activity based model. It was truly a dream team.”
Amazon Web Services is pleased to be part of the team.