U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

 
REPORT
This report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-15-072     Date:  December 2016
Publication Number: FHWA-HRT-15-072
Date: December 2016

 

State of The Practice on Data Access, Sharing, and Integration

 

CHAPTER 5. BUSINESS RULES FOR INTEGRATING AND SHARING

The challenges to accomplishing successful data integration are plentiful but generally fall into two categories—technical and institutional. The first key dimension centers on the technical challenges associated with data systems, including development and maintenance of hardware and software and the specifications for data collection, analysis, and archiving. These are discussed in chapter 3.

Institutional challenges may include centralized policymaking and decentralized execution of those policies; limited appreciation by decisionmakers of the role of data systems in supporting business operations; and lack of formal policies and standards to guide the collection, processing, and use of data within the organization. This chapter describes best practices that may be applied to the development and management of the VDA Framework.

Sources for this chapter include recent documents from the USDOT Real-Time Data Capture and Management State of the Practice Innovations Scan project; NCHRP Report 666, Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies, which addresses the importance of the data management and data governance function within the State transportation departments; NCHRP Report 754, Improving Management of Transportation Information; the USDOT Roadway Transportation Data Business Plan, and private industry.(72,66,73)

DATA MANAGEMENT

Real-Time Data Capture and Management State of the Practice Assessment and Innovations Scan

The Real-Time Data Capture and Management State of the Practice Assessment and Innovations Scan addresses issues related to data capture, data management, archiving, and sharing collected data to encourage collaboration, research, and operational development and improvement.(74) The scan covers five industries: aviation, freight logistics, Internet search engines, rail transit systems, and transportation management systems.

The report includes the following recommendations identifying data management practices and considerations from the five aforementioned industries that are pertinent to the Saxton Data Sharing Framework (pages 2–4):(74)

The scan documented the following best practices for access, security, and privacy (page5):(74)

The scan documented the following best practices for data storage and backup (page 6):(74)

The scan documented the following best practices for operations and maintenance (page6):(74)

The scan documented the following best practice for critical failures (page 7):(74)

Applicability to VDA Framework

All of the data management practices and considerations described above are applicable to the VDA Framework.

Oak Ridge National Laboratory: Best Practices for Preparing Environmental Datasets to Share and Archive

The Oak Ridge report discusses best data management practices that data collectors and providers should follow to improve the usability of their datasets.(75) The report focuses on the preparations for sharing of data, preservation of data, and archiving data. It identifies the following seven best practices for preparing environmental datasets to share:

Each of these practices should be included in any comprehensive data management program.

Applicability to VDA Framework

Each of the seven best practices listed in the previous section (or some form of several of these) could be included in a data catalog for the VDA Framework.

Massachusetts Institute of Technology Libraries

The following Data Management Checklist was designed as a data planning checklist by Massachusetts Institute of Technology (MIT) Libraries for data used in research projects. The checklist is part of a guide, Data Management and Publishing, available from MIT Libraries.(76)

The guide provides the following examples of the types of questions that should be addressed in developing the VDA Framework:(76)

Applicability to VDA Framework

The questions listed in the previous section will help in developing a comprehensive, well-designed data management plan for the VDA Framework by documenting the following important components of a successful data management project:

Policy Analysis and Recommendations for the Data Capture and Management Program: Implementation of Open Data Policies and System Policies for the Research Data Exchange and Data Environments (77)

This report argues that development of the Internet has resulted in a significant global trend to adopt open data and open source policies. Several governmental bodies and not-for-profit organizations around the world are developing initiatives to channel this to harness their benefit. The document provides recommendations concerning emerging open data policies within a connected vehicle research program. It argues that within the transportation industry, an open data policy allows a transformation of the state of the practice by supporting the reuse of data in a collaborative and dynamic framework. Some of the main benefits are the following:

To fulfill the promise of an open data policy, this must be readily accessible and cost effective but at the same time address security, privacy, liability, and quality concerns.

Beyond its open data policy recommendations, this report also outlines the following RDE system policies:

Applicability to VDA Framework

The following conclusions are worth noting.

The VDA Framework should be implemented based on an open data policy. An open data policy is a viable option and is encouraged by the U.S. Government in general and is emerging as a trend with other governments around the Nation and around the world. The level of “openness” is highly dependent on some of the technical inputs—the accessibility of the RDE to public users; the critical and minimum characteristics of the data that will be captured, used, stored, and archived; and the risks/tradeoffs associated with the technical definition of what it means to be open. This report and other related mobility policy reports attempt to apply some definition to these open questions. The whole set of reports and definitions should be vetted by the technical team and stakeholders to ensure that the basis for recommending policies is solid.

The RDE system policies can be based on proven solutions; however, the federation policies require further analysis and development. The alternatives regarding the RDE architecture and set of technologies that are proposed for use in the construction and operation of the RDE appear synonymous with other portals in use with the Federal and State governments, academia, and industry. As a result, most of the RDE system policy can draw from existing models. The key differences, though, from a policy perspective include the wide-scale federation and the monitoring and enforcement of policies throughout such a dispersed system. Once decisions are made about the architecture and technologies, developing a set of alternative models of operation with supporting policies (also referred to as “scenarios”) are a useful next step to determine how the technical, policy, and institutional recommendations align.

State-of-the-Practice and Lessons Learned on Implementing Open Data and Open Source Policies (79)

The document State-of-the-Practice and Lessons Learned on Implementing Open Data and Open Source Policies recommends policies for the DCM and dynamic mobility applications (DMAs) programs. The recommended policies can be summarized as follows:(79)

Applicability to VDA Framework

It is recommended that the policies for metadata, data security, data privacy, intellectual property, liability, and governance be considered in the implementation of the VDA Framework.

Railinc

Railinc processes the Railroad Carload Waybill Sample data each year for the American Association of Railroads as required in statute by the Surface Transportation Board (STB) (which regulates freight railroads). For example, the States all receive annual updates to the Waybill but it is very strictly controlled by the STB. To use the data for studies, one needs to go through a formal approval process to gain access to the data. The data are proprietary and contain origin-destination, tonnage, miles, carrier, commodity (very detailed), and equipment (carload, intermodal). Each year, Railinc also publishes a public waybill sample. Vendors use the waybill sample (also with strict confidentiality) to produce Transearch and other datasets. FHWA uses it to produce the rail elements of the Freight Analysis Framework.

Applicability to VDA Framework

The strict regulation of data may be a useful concept for the VDA Framework.

DATA GOVERNANCE

NCHRP 666, Target Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies

According to NCHRP 666, “Data governance is defined as the execution and enforcement of authority over the management of data assets and the performance of data functions.” (page II-31)(72)

From a practical standpoint, using a data governance model enables the development of standards, policies, and procedures at an enterprise level. A governance model can thus become a focal point where data collection, storage, and use for a particular project or initiative can be set and identified.

From a technical perspective, the use of a data governance framework makes the system more efficient by reducing the number of duplicate data systems, improving quality, and offering better and more coordinated data managing and coordination tools.

The following issues should be addressed in considering a data governance program:

Several models are discussed in the report.

Applicability to VDA Framework

A data governance approach needs to be applied to the VDA Framework.

National Information Exchange Model

The National Information Exchange Model (NIEM) is described as follows:(80)

The NIEM governance framework includes several entities that are similar to the recommended participants in the data governance framework for FHWA Office of Operations. These include an NIEM Executive Steering Council, NIEM Program Management Office, NIEM Communications and Outreach Committee (this would similar to the Communities of Interest), NIEM Technical Architecture Committee, and the NIEM Business Architecture Committee.

Some Federal agencies are addressing the challenge of implementing centralized governance, developing and implementing information exchange guidelines, creating collaborative sharing agreements, and developing enterprise data management maturity, all of which are identified as challenges in the Agency Information Exchange Functional Standards Evaluation report of June11, 2010.(80) These agencies are committed to using the NIEM framework to facilitate the sharing and exchange of information across stakeholder groups (communities of interest). The following example is excerpted from the report:(80)

Applicability to VDA Framework

The use of NIEM or a similar framework supports the exchange of information across nonintegrated databases. It is worth considering for the VDA Framework.

Identification of Critical Policy Issues for the DCM and DMA Programs

This document identifies the following critical policy issues related to governance of open data and open data environments that need to be addressed throughout the DCM Technical Program phases:

Applicability to VDA Framework

The policy issues identified and explored will be relevant to the VDA Framework.

Oregon Department of Transportation

ODOT developed a charter to establish the Transportation Community of Interest Data Council in 2006. The purpose of this council is to identify policy, standards, and processes that support the proper use, management, and maintenance of data assets.

ODOT has recognized the need for strong data governance to create and enforce data management standards. It has established a data management policy, which states the following:(73)

Applicability to VDA Framework

The ODOT data governance model provides an excellent template for defining the roles and responsibilities of all participants in a data governance framework, including the oversight council, communities of interest, data stewards, data business owners, etc. Many of these roles may also need to be defined for the VDA Framework.

Virginia Department of Transportation

In 2008, VDOT implemented a Data Business Plan for the System Operations Directorate to “provide a framework for making decisions about what data to acquire, how to get it, and how to make sure it is providing value commensurate with its cost.”(73) This plan defines a framework of stakeholders and their responsibilities to safely and efficiently manage the data system. This includes data stewards, coordinators, architects, and custodians, as well as business owners and communities of interest. It also defines the roles and interaction within and among data services, data products, applications, business processes, business areas, and business objectives.

Applicability to VDA Framework

The principles associated with a data business plan are directly applicable to the VDA Framework, and the VDOT data business plan framework provides an example of a comprehensive governance structure.

DATA SHARING AGREEMENTS

Data sharing can be crucial in two ways. First, it reduces the need (and associated cost) to collect and manage the same data several times at several offices. Second, it minimizes the risk of giving different responses to the same question that is inherently present when there are several versions of a dataset held by different offices. Formal data sharing agreements are helpful to define how the data will be exchanged across different organizations. Examples can include agreements between Federal and local law enforcement organizations, or between a State transportation department and a department of highway safety and motor vehicles.

One common way to establish a formal data sharing agreement is through memoranda of agreement (MOA) or memoranda of understanding agreed upon by the sharing parties. An example of a MOA is one reached between the Metropolitan Washington Council of Governments and the GIS authorities in various Federal, State, regional, and local organizations with a stake or ownership of data around the Washington, DC, metropolitan area. This MOA, whose purpose was to allow sharing geospatial data among all these parties, included stakeholder responsibilities; the purpose, use, and distribution of the gathered data; liability and other legal agreements; and the terms and conditions of the agreement.(82)

Another example of a data sharing agreement—this time geared for safety matters—can be found in Alaska. The State’s Multi-Agency Justice Integration Consortium includes 20 different agencies, such as the Department of Law and Criminal Division, Association of Police Chiefs, Division of Motor Vehicles, Health and Social Services, Department of Transportation, and Department of Public Safety—all of which were signatories of a MOA “to help agencies more efficiently share complete, accurate, timely information in order to enhance the performance of the criminal justice system as a whole.” Using an automated data collection system called the Traffic and Criminal Software System, this stakeholder group has streamlined the safety and law-enforcement process by making collision, arrest, incident, inspection, and GPS data available to the relevant authorities.(72)

The type of agreement may range from a voluntary collaboration with no binding obligations to one that has enforcement mechanisms. An example of the latter is the data sharing requirement present in the Metropolitan Transportation Commission of the Bay Area in California, which asks local jurisdictions to provide it with regular updates on pavement condition or face the consequence of not receiving Federal grant funds.

Just as external data sharing agreements are extremely valuable in optimizing processes and reducing costs, so is the internal data sharing within a given organization. Internal offices should therefore strive to make data collection and management as unified and streamlined as possible. For instance, within a State transportation department, the data needs for office of transportation statistics and those for the office of transportation safety might be very similar and the two could thus agree to unify their efforts.

Applicability to VDA Framework

Formal data sharing agreements will be necessary in establishing relationships for data sharing for the VDA Framework.

 

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101