Data Consortium Newsletter     

October, 2013 

In this Issue
Nederland uses LiDAR Data for decision-making
DRAPP 2014 update
Colorado Data Summit brings data community together
What is a datum realization and why should we care?
City and County of Denver maps walksheds around transit stations
What can a public entity charge for electronic data in Colorado?
What's the plan for the Regional Data Summit?
Meetings
Contact us
The Data Consortium consists of DRCOG members and regional partners with an interest in geospatial data and collaboration. The Data Consortium Newsletter is designed to improve communication among local GIS professionals and features updates from all levels of government as they relate to data and geospatial initiatives in our region. This newsletter is published quarterly.
 
Some links in this newsletter go to sites maintained by the federal government. These sites are not being maintained during the shutdown; the links should function correctly again once the situation in Washington changes. We apologize for the inconvenience.

Nederland uses LiDAR Data for decision-making

Article provided by Lex Ivey, TerraCognito GIS Services Principal. Lex can be reached at 303-258-3515 or [email protected].

In the mountains of Boulder County, the Town of Nederland has begun using Colorado Water Conservation Board (CWCB) LiDAR for planning and design decision making.  TerraCognito GIS Services has been retained by the town to process the raw LiDAR point data into usable contours and raster surfaces.  The refined datasets will initially be used in mapping and modeling efforts related to the town's infrastructure master plan, but that's just the beginning. The town hopes to fully leverage this rich dataset in fire protection planning, storm water management, economic impact studies, solar radiation potential, and other projects requiring highly accurate mapping and visualization. 
 
While LiDAR data is expensive and not normally available to small towns like Nederland, Mayor Joe Gierlach immediately saw the benefit of using the data resource. Having this highly accurate elevation information is crucial for land-use planning as well as conservation efforts, especially in mountainous areas. Mayor Gierlach and Lex Ivey are optimistic that making use of the CWCB LiDAR data will help streamline public works-related operations of the town, help its Downtown Development Authority with a commercial floor space inventory, and help communicate information to residents when making decisions for the present and the future of Nederland.
Lidar 1
 
Lidar 2
  
Lidar 3 
 
DRAPP 2014 update

The 2014 Denver Regional Aerial Photography Project (DRAPP) is picking up steam. RFPs were released in late August. We received a total of 16 bids: six for imagery, four for DAT, four for LiDAR and two for WMS.

 

During the first week of October, DRCOG distributed the bids to evaluation teams made up of potential DRAPP 2014 partners. This process of having our partners evaluate, score, and interview our potential vendors is essential to the success of DRAPP because it helps ensure the chosen vendor is acceptable to our varied partner group.


Scoresheets are due back to the DRCOG Project Manager Oct. 24. That means there is still time to participate in the evaluation process! Please get in touch ASAP if you would like to review bids from any of the categories (Imagery, LiDAR, Quality Control, WMS/Resale).

 

Based on the closeness of the scores, DRCOG will determine how many of the candidates to interview. We usually bring in the top two from each category.  Interviews will be held sometime between Oct. 28 and Nov. 4. Approval of the vendors by the DRCOG Board will be sought Nov. 20.

 

Our next meeting will be held the morning of Nov. 21. All potential partners should attend. At that point, we will have chosen a vendor, and project costs will be known.  This meeting will be the primary time for you to represent your organization's needs on a few remaining specifications before the final contracts are written with the vendor in January 2014.

 

Also, please be alerted that Letters of Intent (LOIs) will be going out earlier than normal for this project. The Letter of Intent (LOI) process is the means by which DRCOG solidifies the contract with partners and clarifies the amount due. These will be sent to potential partners in late November/early December and need to be signed and returned as soon as possible.

If you have questions or concerns, please contact Ashley Summers at [email protected] or 303-480-6746. 

  

Colorado Data Summit brings data community together

Article provided by Brian Gryth with the Colorado Secretary of State's office. Brian can be reached at 303-894-2200, ext.6213 or [email protected]TATE.CO.US.

The Colorado Secretary of State's Office recently created a Business Intelligence Center (BIC) to "aggregate public data and make it available to the widest audience in the most useful format." To this end, the BIC hosted a Colorado Data Summit in late August to encourage a more cohesive open data community, identify issues and opportunities, and determine next steps for this statewide initiative. Attendance included representatives from cities, counties, regional and state government, non-profits, universities, the health care industry and private technology firms.

 
For those of us in the GIS community, the issues surrounding inaccessible data are well known. Silo-ed or restricted data leads to duplication of effort, as each organization has to mine, "clean," and interpret similar datasets, instead of sharing one amongst themselves.  Due to current data restrictions, each organization in the state that needs this information must request it, pay for it, and prepare it for analysis. This process not only wastes time and money, but creates inconsistencies between datasets and results that are supposed to be comparable.

 
Another identified barrier is lack of a governance structure for open data to guide standardization of data models and associated data products (e.g. metadata). A governance structure could also create a policy framework for data publishing that honors ownership and maintains data integrity.


The group went on to discuss the need for a central repository for data that could serve as a "one-stop-shop" for the State of Colorado. Currently, there are many data portals working toward this goal. The Colorado Information Marketplace, OpenColorado, and the Piton Foundation's Colorado Data Engine all serve out statewide data. DRCOG serves regional-level data through the Regional Data Catalog, and many local governments are beginning to put data on their own websites.

 

While there were many issues and barriers to be discussed, there are equally as many opportunities for success. For example, better illustrating the benefits of open data versus the perceived risks could help build momentum for change. Celebrating the organizations already ushering in change by sharing their own data, and engaging the data user community to provide organizations with feedback on data usefulness are other strategies that were discussed.

 

Next steps for this group include meeting again in a few months to continue this discussion and to make an action plan.  

What is a datum realization and why should we care?
Article written by Ashley Summers, DRCOG GIS Manager, inspired by Pam Fromhertz, NOAA NGS Colorado Geodetic Advisor. Ashley can be reached at 303-480-6746 or [email protected]; Pam can be reached at 303-202-4580 or [email protected].

In the world of GIS, we understand the importance of specifying the correct projection and datum of our data. To neglect this could cause misalignments or distortions that lead to error in our analyses.  Even though we know this in theory, how many of us dig deeper into the accuracy issue and ensure that we are specifying the correct Datum Realization, Epoch and Geoid for all our datasets and capturing that in the metadata? Probably not many.  That may need to change in the near future because with increasing capabilities of our data collection technologies (e.g. LiDAR) and more demanding project requirements (e.g. to support survey-level work), failing to understand and address these components could lead us down a rabbit hole of errors.

 
For those of us who haven't been in school for while or those who picked up GIS on the job, here's a little refresher on terms.

 

Datum mapAn Ellipsoid is a mathematical surface. Its dimensions are determined as a best fit to the world's geoid model and allows for easier calculation of positions.

 

The Origin in the past (e.g. for NAD27) was a point at Meades Ranch, KS based on the Clarke ellipsoid of 1866. The origin for more recent datums (e.g. NAD83 and WGS84) is the center of the earth's mass based on the Geodetic Reference System of 1980 (GRS 80).

 

Image from http://www.spatial-ed.com

 

A Horizontal Datum refers to the reference specifications of a system of coordinates. The datum will specify the ellipsoid and the origin.     

 

Datum Realization (also referred to as a Datum Adjustment) occurs when the National  Geodetic  Survey (NGS) re-computes the positions of  known points (i.e. survey marks) based on new data observations to account for technological improvements and  movements across the earth's surface.

 

An Epoch is the date that the coordinates were held in the realization. An epoch is simply an extra level of granularity used to provide a time stamp of the positional information. This has become more critical in places that are very tectonically active (Note that Colorado is fairly tectonically stable, so we don't have to worry much about epochs). 

 

A Geoid represents the gravity potential that approximates mean sea level.

 

To summarize those terms, let's dissect this coordinate system description:

 

NAD 83(2011) epoch 2010.00.

  • NAD83 - North American Datum of 1983. This is the geodetic datum. It references the GRS80 ellipsoid.
  • (2011) - The year of the latest NGS datum realization.
  • epoch 2010.00 - The time stamp of when the observational data computations were held.

Now the big question: Why do we care about these things?

 
Answer: Because it can affect the accuracy of our work. 

NGS has made four realizations to NAD83.

  1. The first realization was released in 1986 before GPS (so it was 2-dimensional). The relative national network accuracy was at the 1m level.
  2. The second realization was done on a state-by-state basis due to the advent of GPS, and exceeded the accuracies published in the 1986 realization. This realization occurred in Colorado in 1992. It is referred to as the High Accuracy Reference Network (HARN) realization and it increased accuracy nationally tenfold; to the 10cm level. Relative accuracy was actually at the 2 cm level here in CO from the HARN.
  3. The third realization was released in 2007 - called NAD83 (NSRS2007) - and was based on approximately  70,000 monuments. The data from GPS observations on the monuments were tied to 700 stations in the Continuously Operating Reference Stations (CORS) network - called NAD83(CORS96). Due to the continuous GPS data collected by CORS, the nationwide accuracy level increased to 1cm, and again,  Colorado remained at a relative accuracy at the 2 cm level. (Keep in mind that this realization, as well as the first two, has been officially superseded).
  4. The fourth realization was released in 2011 - called NAD83(2011) epoch 2010.00. Observed data at approximately 80,000 NGS monuments were tied to  1195 CORS (IGS08)  epoch 2005.00 and produced better absolute accuracy, both horizontally and vertically. If you use any of the NGS products or tools (e.g. Opus, NGS CORS data) it is best to use this realization to ensure consistency.

 

For much of the analyses that we do in government GIS, we are looking at data trends at a less-detailed level. Plotting incidents of crime, estimating canopy cover in urban landscapes, and determining a walkshed to a train station don't rely on sub-centimeter accuracy.

 

However, when you are building bridges, putting in water lines, or working with planimetric data (e.g. Edge of Payment or Building Outlines) the difference between 1cm, 10cm, and 1m could be substantial. It all comes down to what you are trying to do with your GIS data.

 

The most important concepts to take away from this article are: 

  1. Consider updating your datasets to the most recent datum realization to stay in sync with NGS.  If you can't do that, then...
  2. Document, document, document the full projection and datum for all your datasets and data sources so that you and all users of your data know the level of error when the data is applied.

How to apply this knowledge in ArcMap:
Most of us use a State Plane projection in US Feet - either North or Central, so that leaves us with the following eight options when mapping our data. Here's how we think they correspond to the realizations listed above.*
  

 Realization

Read about the differences between CORS96 and NSRS2007 here.
Download NAD 1983 NRS2011 projection files here

 

*Pam, Ashley, and ESRI Technical Support collaborated on crosswalking these projections to their corresponding realizations. With that said, the authors have not yet found official documentation that confirms these assertions. 

 

Next Steps
We will be featuring more articles and presentations on this topic through the Data Consortium over the next several months. In the interim, if you have specific questions or want more information on this complex topic, please contact Pam Fromhertz at NOAA. 

 

City and County of Denver maps walksheds around transit stations

Article provided by Andrea Santoro with the City and County of Denver. Andrea can be reached at 720-865-2946 or [email protected].

As the population of the Denver metro area continues to grow, improving mobility and transportation infrastructure is key to our region's success.  We have seen the commitment to an expanded rail transit system with the Regional Transportation District's FasTracks program and the recent opening of the West Rail Line, connecting downtown Denver to Golden.  As this system is being constructed and stations are established, transit area planning and Transit-oriented Development (TOD) have become a high priority for the City and County of Denver.  Denver's Community Planning and Development Department (CPD) has taken the lead on this by maintaining a TOD Strategic Plan and adopting numerous station area plans specific to individual stations.

 

TOD is defined as the development surrounding a transit station, where the station is a key feature.  TOD areas ideally contain high-density and a mix of land uses, where people can live and work and move around without dependency on an automobile.  They are generally identified by their walkshed, which covers the distance it is assumed that people will walk to get to a transit station.  For light rail and commuter rail, it is estimated that people are willing to walk approximately half a mile.  In the past, CPD has mapped TOD walksheds "as the crow flies," which does not necessarily represent the area where people are physically able to walk.  To produce more accurate representations of the transit station walksheds, CPD's GIS staff utilized Esri's Network Analyst to map a half-mile distance against a walk network, incorporating off-street trails, and taking into account barriers such as interstates, rivers, and railroads. 

 

Walksheds mapThe process of mapping the walksheds began with preparing the base data, or the walk network, against which the analysis would be run.  The street network was modified to exclude streets where people do not walk, such as highways and highway ramps.  Pedestrian bridges and off-street trails were added in, as well as future connections and network intersections.  The dataset is populated with key attributes for distance, walk speeds, and time traveled, which allow the software to map all possible half-mile routes traveling away from each station. An area is generated by connecting the half mile routes, creating the walkshed polygon.   This map of Perry Station shows the traditional half-mile buffer as compared to the modified walkshed derived from the walk network. By mapping the actual half-mile walksheds, CPD is able to assess connectivity, identify barriers, and evaluate where potential infrastructure improvements would be most beneficial, and planners are able to more effectively plan for future development
 

What can a public entity charge for electronic data in Colorado? 
Article provided by Dave Murray with the City of Westminster. Dave can be reached at 303-658-2140 or [email protected].
I posed this question to one of our staff attorneys a few months back.  There was a landmark case out of California this year; The Sierra Club vs. Orange County, which required the county to produce the records requested not to exceed the cost of duplication.   Cost recovery could not be applied by the county and especially not to the tune of $475,000, which was the price of their landbase.  While this case was out of California and not Colorado, legal rulings out of California can influence similar cases here.

So what does Colorado permit a public entity to charge under the Colorado Open Records Act (CORA) for electronic data? 
  1. Nominal research and retrieval fee for the information requested.  A fee of $15 to $20 per hour has been upheld as a reasonable fee in Black v. S.W. Water Conserv. Dist., 74 P.3d 462.
  2. The actual cost of manipulating data in order to generate a record requested by a person.  CRS 24-72-205 (3).
  3. If the public record is a result of computer output other than word processing, the fee for a copy, printout, or photograph thereof may be based on recovery of the actual incremental costs of providing the electronic services and products together with a reasonable portion of the costs associated with building and maintaining the information system. Such fee may be reduced or waived by the custodian if the electronic services and products are to be used for a public purpose, including public agency program support, nonprofit activities, journalism, and academic research. Fee reductions and waivers shall be uniformly applied among persons who are similarly situated. CRS 24-72-205 (4)
  4. All costs associated with records transmission but no fee for sending digital records via email. HB 13-1041.

In the case of a public entity that is charging for electronic data, the options suggested by our staff attorneys are:
Option 1.  Work with Data Consortium members to persuade the charging entity that the data is being used for a "public purpose" and request the fee be waived per CRS 24-72-205 (4).

 
Option 2.  Inquire as to other entities receiving this information from the public entity at no charge and convince the entity that the Data Consortium members are "similarly situated" as describe in CRS 24-72-205 (4).


Option 3.  Ask the public entity the basis for the fees they charge so we can determine if the fees are consistent with CORA.

 

Option 4.  If legally defensible, challenge the fee schedule in court.

 

The Data Consortium's mission is to provide members with the most current authoritative information to serve our customers.  When some jurisdictions do not participate, we all lose.  We are left scrambling to come up with data to fulfill our roles.  This is why the effort to acquire data for public purposes is so important. 

Moving forward, I suggest we consider Option 1 when faced with government entities that charge for their data.  A collective effort at making a formal request to waive fees for those of us in the public sector could be an effective course of action.  Hopefully these entities would come to realize the greater benefit of sharing data and amend their current policies. 

 

Options 2 and 3 would be the next steps to help us understand the scope of how data is being sold and to whom.  As the CORA states, a portion of the cost of creating the data can be recovered.  Do charging entities have a review schedule for their fees like many private- sector companies?   Also, what constitutes a case where data is freely given to government entities and is it consistent with CORA?  While jurisdictions like Denver and Broomfield have given up charging for digital data, what  revenue is being  generated by those that do?  These are just some of the questions that would help us make a better case for data sharing. 

 

The last option is not viable at this time.  Court proceedings are expensive and time-consuming.  Still, if it comes to that, there must be enough participants willing to fund an effort to make the case worthwhile.   If you think of all the time wasted in duplicate effort, the economics become more favorable. 

  

This is an ongoing effort that will discussed and followed up on at future Data Consortium meetings.  If you have any comments or would like to discuss this effort, please contact Dave Murray at 303-658-2140. 

 

Additional reading:
21st Century Sunshine: Modernizing CORA

What is a reasonable CORA fee?
Changing technology landscape requires changing open data policies

What's the plan for the Regional Data Summit?
DRCOG hosted its largest Regional Data Summit to date last January. We had excellent attendee turnout, and a full day of thought-provoking presentations and discussions. Although DRCOG staff were pleased with the event, the post-event survey showed mixed results on how to move forward. The Regional Data Summit premiered in 2009 as a way to highlight the accomplishments of the Regional Data Consortium - a group of GIS professionals in the region working "to support informed decision making in the Denver region by organizing, developing, maintaining, sharing, enhancing, and distributing regional data." As many of our regular Data Consortium meeting attendees know, this mission is challenging. Collaborating to create authoritative, regional data takes staff time, executive support, agreement on methodologies and schemas, and consensus on a data sharing policy. As the Data Consortium coordinator, I am happy to report that we are beginning to make progress on some of these components by encouraging a more open and collaborative Data Consortium meeting format and by issuing a quarterly newsletter that highlights our successes and challenges. With that said, there is a considerable amount of work to be done to get traction and until we make further progress, DRCOG is postponing the Regional Data Summit. When we meet again for this celebration, the role of the Data Consortium and the purpose of the Data Summit will be clearly defined and set up for success!
 
Meetings
b.  Next DRAPP meeting is 11/21/13
c.  Next DRDC meeting is 2/27/14
 
Contact us
For more information on any of the topics mentioned in this newsletter or if you have an idea for an article, please contact DRCOG GIS Manager Ashley Summers at 303-480-6746 or [email protected]
  
  
Disclaimer: The information provided in this newsletter is compiled from multiple sources and is intended for informational purposes only.  DRCOG assumes no responsibility or legal liability for the accuracy, completeness or usefulness of any information in this newsletter.