Exercise 6: Geocoding and normalization of data

Goal:

The goal of this project was to illustrate to our class the difficulties that can arise when trying to normalize data that is collaborated form outside sources. It was also designed to show how challenging geocoding can be depending on the information provided in the data set. For the assignment we where tasked to normalize a set of mine addresses given to us through a Wisconsin DNR dataset. The original dataset had come to us in a way where we needed to extract key features from the dataset like the mine unique id field, the mine name, and any information given about the address. In some cases it was a street address other times it was only stated using the PLSS form of surveying. In either case we found that we needed to put a lot of effort into keeping consistent extraction into our own designed tables (figure 1.1 will show the normalized table). Once normalized we could then create a shape file which could later be used to geocode to the actual location. Finally we then needed to compare our results to our classmates to see how much variation occurred.

Methods:

To start the assignment we first needed to normalize the data. For most of us we decided to use a similar format as the one shown to us in the demonstration by Dr. Hupy. We where also advised to create a separate column for the PLSS address making it easier to find and be able to portray that information if needed. We then needed to manually go through each unique mine id and find all of the mines that had been assigned to us with a code. Once acquired we just needed to type in all information provided to our table and complete the table. The final product is shown below.

Figure 1.1: normalized table of the 21 mine sites given to me to geocode from the WDNR table

After we then could extract our table into Arcmap to produce a shape file. With this shape file now imported into Arcmap we then needed to geocode all of the addresses. To accomplish this we first needed to open our geocoding toolbar. From there we could process our table. With the original process I received 14 matches one tie and 6 unmatched. Now although this seems like a good start I was quick to find out this was not the case. From here we needed to use a lot of mental skill and tools to find the actual mine. To start I first went to all of the mines that did not match and needed to place a location of the mine. I used the PLSS finder and a imagery base map to find the locations I thought where the mine sites. To accomplish this I would enter all of the PLSS information into the database and then get an idea of where the location may be. After by overlaying the PLSS quarter quarter sections onto my map over the imagery I was able to get within the area of the mine and visually look for the location to match an address. Although this only solved the issues for the unmatched mines. I then went back through all of the mines to make sure the where properly placed. I found that 15 of my 21 mines where not in the right location and needed to be manually moved in the same process.

After the geocoding was completed I then could extract my properly placed points into a shared file for all of our classmates to use.

For the next objective we where then asked to compare our results to two things. We where asked to find the variation between groups to see how much our mine sites varied and we where also to compare how far off our personally placed mines where to the actual mines. I had found the distance variation between groups and between my own mines and the actual using a couple of steps. First I needed to query out all mines that where not applicable to the situation and queried all mines that had the same mine id as my own mines. With the new selected feature class I was able to then project both sets into UTM to make sure the units where in meters and run a near tool to find the distance from each map to the nearest mine between groups and mine in one data frame and between the actual site and my own in another data frame.

Results:

After running the queries and the near tools the results where shocking. I had found that only a few of my mine location had been close to my classmates groups by distance of meters. However, majority of them where significantly off. This was perplexing to me since we where all given the same datset and yet we where that far off in our findings. For the second near table I was a little closer I had majority of my mines near the actual mine, however, there was about 7 mines that I was not even within the right ball park of correct location. I am not sure as to why this may be the case, might have to do with outdated bsaemap or with improper data normalizing. Either way the results are shown below in figure 1.2-1.4.




Figure 1.2: location of my maps in comparison to the other three classmates with the same location geocoded.

Figure 1.3: near table tool showing the closest mine synced with distance away in meters

Figure 1.3: comparison of actual miens to my personally geocoded mines

Figure 1.4: displaying the distance from my mine locations to the actual mine in meters.



 Discussion/Conclusion:

This lab was very helpful to my skills in GIS I was bale to witness first hand the complications that can arise from improperly entered data, and how normalization of a dataset can be key to display of data and geocoding in future applications. Since you are not always given consistent data or data that is complete this lab is a great representation of real world issues that can arise in a job, field work. It developed and introduced all of us to the skills we will need and the mind set we need to have when setting up data collection to save from future headaches of normalizing.
There was a few errors that had arises throughout this activity that are worth noting. First was the inherent error that is very relevant to this lab. If the person who had collected the data would have used more domains and subtypes along with creating a few more columns for some of the information the data would not have needed to be normalized to the extent that it was by us. Also With the class being trained through a demo on how to normalize and some tips on how to arrange the data you can say there was some gross error involved later on when some groups didn't have a mine unique value and the data needed to be manipulated to be compared in the near tool.

I would have to say if we where asked to achieve a hundred percent accuracy with this data and be hundred percent consistent between our classmates placement we would have needed to communicate and agree on a standard way of normalization of data along with acquiring a more consistent and completed data set that would have provided better location for the actual mine site.

Altogether the activity was very useful and helpful in understanding all of the complications that can arise from data normalization and inherent error. I also enjoyed being able to work with other classmates to see how each person can have a different view on the same task. I always appreciate seeing the work of fellow peers for it allows me to asses my own efforts and skills and see what I need to improve on and what I can capitalize on and help others improve in their own skills.