Thursday July 16, 2020:
Terminal one-liner to create shapefile from WKT using ogr2ogr
Fair warning, this is a Linux-themed solution. But I expect it could be ported to Windows without too much trouble. Today I needed to create a shapefile with a single point geometry in it to use as an input for a random GIS utility executable, which required its input to be a an actual shapefile rather than an array of coordinate pairs, or even a single coordinate pair, or even separate x=/y= input parameters for a single coordinate pair. I already had the point coordinate I wanted to use, and I didn’t want to go to the trouble to make this shapefile! So I got to wondering if I could use ogr2ogr [...]
Tuesday May 21, 2019:
CentOS 7 and PostGRESql: Move the PostGRESql data_directory
Long time no see. Scroll down for the answer, or start here for the story.. Today, by happenstance while testing a backup/restore strategy, my coworker discovered that our CentOS 7 installation from ISO/DVD used a default disk partitioning strategy that allocated a majority of disk space to the
Tuesday March 17, 2015:
Socket-served Internet mapping, anyone?
I may or may not be on to something. For awhile I’ve been growing more and more tired of the typical Internet mapping user experience. It’s basically this: open a map and zoom/pan somewhere, wait a moment while (hopefully) some “working” graphic gives you a clue that the map hasn’t forgot about you; meanwhile, some basemap tiles load, then after a moment or two, “poof!” all the interactive data (i.e. the sole reason you are using that particular map) appears all at once. Fortunately, in most cases, the basemap tiles “sprinkle in” one by one as they become available, which does give the user an impression that something is happening. But more often than not, basemap tiles [...]
Saturday March 07, 2015:
Q: What’s The best way to convert CDs to MP3s in Windows? A: VLC and the Command Line. Here I’ll show ya..
Today my friend (shout out to Nic Zamora) wanted to know if I still had a recording we made of a song he co-wrote way back in ancient history ..like 2007 or something. Gah.. Well it got me to thinking—my other buddy (shout out to Corey Nolan) had a similar request awhile back for a recording we did with his brother, Dustin, around the same time. The problem was, the only recording I had of the latter was on CD. Since then, that one simple deterrent—being on CD—was solely responsible for preventing me from ever “getting around to it”, like I said I would. The reason I hate ripping CDs is because it’s such a ridiculous process ..first you have to open some asinine software (and God help you if it’s Windows Media Player), click around like a fool, be confused, drag files and/or locate folders, agonize over a variety of settings, many of which you, A) may not know anything about, 2) remember from the last time you did this resulting in inconsistently ripped music, and D) ..worst of all, potentially but unintentionally introduce some weak-sauce copy protection. [...]
Thursday January 22, 2015:
Gabriel Weinberg’s PostgreSQL Tips and Tricks
Here’s a solid article on tuning your PostgreSQL instance and optimizing your indexes. Also, here’s a nice primer on creating and managing indexes in PostgreSQL. I wanted to share ’em now and save ’em for later. /E
Tuesday November 25, 2014:
ArcPy CalculateField_management Complex Python Expression to Return a String
My coworker was having some trouble implementing a CalculateField expression in Python so I wanted to document the solution for anyone with a similar issue. (Note: If you just want the answer, see the What ultimately worked.. heading, below.) Being unfamiliar with ArcPy, I was intrigued by this idea of defining a function as a string value, then passing that string as a variable into another function. I can see how that opens a powerful door—but it also strikes me as ultra weird, and because Python has some very particular spacing/indentation rules (as compared to say, JavaScript), I figured this technique would be ultra-prone to syntax issues ..and thus could be especially difficult to troubleshoot. So in this case, I also wanted to demonstrate how I ultimately thought through the problem. [...]
Friday September 26, 2014:
Create UTFGrid Tiles from PostGIS Tables
I assume you’re where I was about a week ago. That is, you’ve heard of UTFGrid, and now you want to render your own UTFGrid tiles. Perhaps you found yourself looking at this thread over at GIS.SE, but you don’t want to jump into TileStache just now. Anyway if you’ve got a working installation of GDAL/OGR and Mapnik 2+, complete with Python bindings, I’ll show you what worked for me.. Because this is merely an adaptation of Matthew Perry’s original solution, I highly recommend considering his original blog entry on the topic, complete with discussion points and caveats, before proceeding further! [...]
Wednesday September 24, 2014:
OGR VRT: Connect to PostGIS DataSource
I needed an OGR VRT for something and didn’t find a clear example on the web all in one place, so here goes. Somewhere on your system, create a new file with a
Monday September 22, 2014:
MySQL Implementation of Google’s Encoded Polyline Algorithm
I just noticed someone created a MySQL implementation of Google’s Encoded Polyline Algorithm incorporating some elements of my PostgreSQL/PostGIS approach, specifically, support for multi-part, multi-ring polygon geometries. And this is exciting, at the bottom of his post, Fabien says he’s working on a solution for consuming the Google encoded geometries in Leaflet. Nice! I love open source software! I thought I’d mention the MySQL approach here to help expose Fabien’s solution to English searches and drive a little more traffic to his site. If you can’t read French, Google Translate is your friend! [...]
Friday June 06, 2014:
Convert Google Maps Polygon (API V3) to Well Known Text (WKT) Geometry Expression
There’s dozens of reasons why you might want the Well Known Text (WKT) geometry expression for a Google Maps Polygon object. Assuming you’re using the Google Maps API V3, and you’ve got a variable referencing your Polygon, I’ll suggest two approaches you can take to iterate over the paths and vertices in your Google Maps polygon and return the geometry expression as a Well Known Text string. Add a Simple Utility Method to Your Project Easy enough. Just add the following method to your project. Look below the method for an example of how you’d call it.
Monday January 27, 2014:
PostGREsql/PostGIS Implementation of Google’s Encoded Polyline Algorithm
[Edit 30 Jan, 2014] I added an additional PostGREsql method to perform Polygon encoding by concatenating polygon geometries (delimiter: †) and their inner rings (delimiter: ‡) together into one massive encoded block of ring features. I also provided an example JavaScript method demonstrating how to bring the amalgamated polygon feature encodings into your Google Map. By some uncanny twist of the fates, I’ve elected to use, had to use, and/or been asked to develop applications that use Google Maps ASCII Encoded Polyline expressions. In previous encounters, I’ve used a PHP class to handle the encoding task, and most recently I wrote a [...]
Friday November 08, 2013:
PostGIS: query all multipolygon parcels with at least one hole
I was writing some code to iterate over Well Known Text expressions for polygon features, and I decided I needed to test the most complex edge-case I could think of–multipolygon geometries where at least one of the bound polygons has a hole (i.e. an interior ring). I ended up with the following query. This seems like the kind of thing I’ll want to reuse later, so I’m noting it here. For good measure, I also use a rudimentary technique to sort the output with the most complicated geometries in the table at the top of the list. Basically, the more “text” it takes to describe the geometry using Well Known Text, the larger and more complex I figure it must be! [...]
Monday August 05, 2013:
Install httplib2 to your Preferred Python Runtime after ArcGIS Unceremoniously Hijacks the First Attempt
We’re going to use Google Maps Engine for some stuff, and so I thought I’d throw my first codes at it using Python. Well.. Google’s Python example requires the httplib2 library, so I needed to install that. When I did—following the library’s download/instructions—for some reason, the library went into service against ArcGIS’s embedded Python runtime, rather than my favored 2.7 instance, which I use for every single thing. ::grimaces:: To fix this, I just ensured the new library’s setup.py script was called from my preferred Python runtime rather than let Windows, or some black magic, decide which instance it should integrate with httplib2. [...]
Friday December 28, 2012:
osm2pgsql help and usage
I wanted a more convenient place to read the osm2pgsql help output, so this seemed as good a place as any for it. /E
Friday December 21, 2012:
Resident “lunatic” of LinkedIn
Just updated my LinkedIn profile.. This is just TOO FUNNY right now! [image] I totally had to save this for posterity. Unfortunately it will probably disappear before you can say “Jack Robinson”. 8)
Tuesday December 18, 2012:
mapnik rundemo.exe error: run from within the demo/c++ folder?
So I just installed the mapnik 2.0.1 binaries for Windows, and I ran into a “gotcha.” I couldn’t figure this out by Googling, so hopefully this post will help someone. (Be sure to note the embarrassing conclusion.) Specifically, following a fresh mapnik install, rundemo.exe told me this:
Tuesday December 18, 2012:
osm2pgsql and windows errors: failed to start MSVCR90.dll, Connection to database failed, etc..
I’m following the BostonGIS tutorial(s) to learn how to setup an OpenStreetMap tile server on Windows (XP 32, cos’ that’s what’s on the desk), and I’m running into headache after headache. So this post notes the gotcha’s I’m encountering and how I’m fixing them (optimistically assuming I fix all of them). “Thar be dragons..” 8) ————————————————— Part I: “Failed to start MSVCR90.dll” First, as recommended I tried using the HOTOSM installer, but when I launch it using the example command provided on the BostonGIS site, I got the error above. I’d logged out/in, rebooted, ad nauseam, to no avail. Also the problem can’t be my redist libs–this install of XP is only a couple months old, and I have all the redists libs I know of (2005, 2008, 2010—plus, when I try reinstalling a redist it says I have the latest). I consider uninstalling and reinstalling Visual Studio to be more trouble than just trying something else to get the OSM download into PostGIS, so.. [...]
Saturday October 20, 2012:
Decode Google Map encoded points as Well Known Text (WKT) with Python
I had close encounter of the 5th kind yesterday.. here’s the gist.. It started when someone “gave” me a GIS dataset (..of polygons, kind of..) that a colleague of theirs, way back in ancient history, chose to pre-cook as ASCII-encoded point pairs. Their intention was almost certainly to use the pre-cooked data in Google Maps. Anyway, being arguably sane, I wanted to return this data to a more GIS-normal format so I could put it in a database like MySQL or Post and use it for other stuff. I considered a few different approaches to this problem, including creating a Google Map that could load-in all of the polygons from their encodings, then iterate over the polygons, interrogate the polygon point pairs, and finally concatenate WKT features from the points and save the geofeatures into a MySQL table. This approach offered the advantage of using Google’s existing Maps API to do the decoding for me. But let’s face it, that’s lame, uninspired, and not inventive.. it wasn’t even interesting. [...]
Saturday January 21, 2012:
PostGIS: count all features of each GeometryType in a spatial table
Sometimes, just when you think you’ve got something figured out –you get reminded that you really don’t. :/ As you may know, ESRI allows for single and muli-part geometries to live in the same FeatureClass. So, if I have a shapefile of roads, there might be both LINESTRING and MULTILINESTRING features in that dataset. I live just loose-enough not to care about that. But I do need to be aware of it when I’m cobbling data in PostGIS. In thise case, I was getting a PostGIS error tying to do a Dissolve-By-SQL, so I thought why not get a quick count of each GeometryType in the dataset? Maybe I was running into issues single and multi-part geometries were blurred together. It took me an embarassing chunk of time to get this right, so I figured I’d post the recipe in case I needed a reminder later. [...]
Saturday November 19, 2011:
Prepare a Shapefile for OpenScales using ogr2ogr and PostGREsql
This post explains how to import GIS data (a shapefile, in this case) into a database (PostGREsql) so it can be consumed by most any mapping API. I have OpenScales in mind, but this approach will support any mapping app with functions for rendering feature overlays using geodetic coordinates (i.e Longitude and Latitude). In many cases, you’ll need to translate your feature data out of a projected/cartesian system and into a geodetic/spherical system; so I’ll include a demonstration of that.
Friday November 18, 2011:
ogr2ogr: Export Well Known Text (WKT) for one feature to a CSV file
Perhaps you’re looking for this?
Sunday November 06, 2011:
OpenScales: Coordinate Transformation from Uncommon Projections
———— UPDATE 2.19.2013: It turns out you don’t have to rebuild OpenScales to apply uncommon projections, you can just assign the projection directly to ProjProjection.defs. (See Simon L.’s answer here in the OpenScales user group.) So if I modify my example toward the bottom of the post (at #5), here is how I would apply the alternate, and arguably better, approach. (Fair warning, I haven’t compiled this and tested it yet, the only difference is the first line), but this is how I expect it should be. If it doesn’t work like this, it’s likely ‘cos an instance of ProjProjection needs to be created first, and the defs property of the instance needs to be appended. In other words, if this doesn’t work, follow the line instantiating moNADProj with moNADProj.defs[‘EPSG:102697’] = “+title…. blah blah”; One of these days I’ll make super-sure this is correct. But I didn’t want to forget where I found this, hence updating the post.) [...]
Saturday November 05, 2011:
Nikon Coolpix 3100 Near Infrared Hack
While this post is not in-theme with the stated goals of the blog, I wanted to save my notes somewhere as well as make a URL to show-off for my friends. On the other hand, I’m opimistic my experiment will enable some “eco-minded research”, so perhaps there’s still a loose tether linking this back to the GIS community.. Basically, I hacked my old, 3 megapixel Nikon Coolpix 3100 today so that I could take NIR (near infrared) pictures. The following images serve as a storyboard for the process. 0: The tools and materials involved include (clockwise from bottom-left) —-> small round-nose pliers (my wife uses these for beading) [...]
Monday October 17, 2011:
Install GDAL on Windows
Later posts on this blog will assume a working install of GDAL/OGR, so it’s prudent that I first demonstrate how to get a fresh cut of GDAL/OGR up-and-running on a Windows system. Windows users have a few options for installing GDAL (see this question at gis.stackexchange,which will point you to Christoph Gohlke’s binaries –scroll down to GDAL, and the OSGeo4W Installer, among other approaches). However, I’ll guide you through what works for me, which can be summarized as follows: 1) Install Python 2) Install the GDAL binaries published by Tamas Szekeres 3) Append your environment Path variable [...]
Sunday October 16, 2011:
Google Maps for Flash/Flex: Dead Man Walking
“When life gives you lemons..” Let’s just review the signs:
Terminal one-liner to create shapefile from WKT using ogr2ogr
Fair warning, this is a Linux-themed solution. But I expect it could be ported to Windows without too much trouble. Today I needed to create a shapefile with a single point geometry in it to use as an input for a random GIS utility executable, which required its input to be a an actual shapefile rather than an array of coordinate pairs, or even a single coordinate pair, or even separate x=/y= input parameters for a single coordinate pair. I already had the point coordinate I wanted to use, and I didn’t want to go to the trouble to make this shapefile! So I got to wondering if I could use ogr2ogr [...]
Tuesday May 21, 2019:
CentOS 7 and PostGRESql: Move the PostGRESql data_directory
Long time no see. Scroll down for the answer, or start here for the story.. Today, by happenstance while testing a backup/restore strategy, my coworker discovered that our CentOS 7 installation from ISO/DVD used a default disk partitioning strategy that allocated a majority of disk space to the
/home mountpoint, and a dearth of space to /, the root mountpoint. In our example, this meant about 2.1 TB available to the home mount and appx 50 GB to the root mount.
Meanwhile, our default PostGRESql 9.x install on this platform established the database instance to a location on the root mountpoint, which naturally limited it to 50 GB of consumption. While 50 GB [...]
Tuesday March 17, 2015:
Socket-served Internet mapping, anyone?
I may or may not be on to something. For awhile I’ve been growing more and more tired of the typical Internet mapping user experience. It’s basically this: open a map and zoom/pan somewhere, wait a moment while (hopefully) some “working” graphic gives you a clue that the map hasn’t forgot about you; meanwhile, some basemap tiles load, then after a moment or two, “poof!” all the interactive data (i.e. the sole reason you are using that particular map) appears all at once. Fortunately, in most cases, the basemap tiles “sprinkle in” one by one as they become available, which does give the user an impression that something is happening. But more often than not, basemap tiles [...]
Saturday March 07, 2015:
Q: What’s The best way to convert CDs to MP3s in Windows? A: VLC and the Command Line. Here I’ll show ya..
Today my friend (shout out to Nic Zamora) wanted to know if I still had a recording we made of a song he co-wrote way back in ancient history ..like 2007 or something. Gah.. Well it got me to thinking—my other buddy (shout out to Corey Nolan) had a similar request awhile back for a recording we did with his brother, Dustin, around the same time. The problem was, the only recording I had of the latter was on CD. Since then, that one simple deterrent—being on CD—was solely responsible for preventing me from ever “getting around to it”, like I said I would. The reason I hate ripping CDs is because it’s such a ridiculous process ..first you have to open some asinine software (and God help you if it’s Windows Media Player), click around like a fool, be confused, drag files and/or locate folders, agonize over a variety of settings, many of which you, A) may not know anything about, 2) remember from the last time you did this resulting in inconsistently ripped music, and D) ..worst of all, potentially but unintentionally introduce some weak-sauce copy protection. [...]
Thursday January 22, 2015:
Gabriel Weinberg’s PostgreSQL Tips and Tricks
Here’s a solid article on tuning your PostgreSQL instance and optimizing your indexes. Also, here’s a nice primer on creating and managing indexes in PostgreSQL. I wanted to share ’em now and save ’em for later. /E
Tuesday November 25, 2014:
ArcPy CalculateField_management Complex Python Expression to Return a String
My coworker was having some trouble implementing a CalculateField expression in Python so I wanted to document the solution for anyone with a similar issue. (Note: If you just want the answer, see the What ultimately worked.. heading, below.) Being unfamiliar with ArcPy, I was intrigued by this idea of defining a function as a string value, then passing that string as a variable into another function. I can see how that opens a powerful door—but it also strikes me as ultra weird, and because Python has some very particular spacing/indentation rules (as compared to say, JavaScript), I figured this technique would be ultra-prone to syntax issues ..and thus could be especially difficult to troubleshoot. So in this case, I also wanted to demonstrate how I ultimately thought through the problem. [...]
Friday September 26, 2014:
Create UTFGrid Tiles from PostGIS Tables
I assume you’re where I was about a week ago. That is, you’ve heard of UTFGrid, and now you want to render your own UTFGrid tiles. Perhaps you found yourself looking at this thread over at GIS.SE, but you don’t want to jump into TileStache just now. Anyway if you’ve got a working installation of GDAL/OGR and Mapnik 2+, complete with Python bindings, I’ll show you what worked for me.. Because this is merely an adaptation of Matthew Perry’s original solution, I highly recommend considering his original blog entry on the topic, complete with discussion points and caveats, before proceeding further! [...]
Wednesday September 24, 2014:
OGR VRT: Connect to PostGIS DataSource
I needed an OGR VRT for something and didn’t find a clear example on the web all in one place, so here goes. Somewhere on your system, create a new file with a
.ovf extension. Inside that file, add some XML like the following to define your PostgreSQL connection:
That name=”WKTGrid” is semantically unrelated here. I have been experimenting with including WKT geometry data in UtfGrid tiles, and that name is relative to my experiments. You can provide most any value for name. However, do note that the layer name is referenced in the ogrinfo command.
[...] PG:host=127.0.0.1 user=postgres dbname=gis password=l00per parcels_cama_20140829_pmerc SELECT tms, owner_name, the_wkt FROM parcels_cama_20140829_pmerc
Monday September 22, 2014:
MySQL Implementation of Google’s Encoded Polyline Algorithm
I just noticed someone created a MySQL implementation of Google’s Encoded Polyline Algorithm incorporating some elements of my PostgreSQL/PostGIS approach, specifically, support for multi-part, multi-ring polygon geometries. And this is exciting, at the bottom of his post, Fabien says he’s working on a solution for consuming the Google encoded geometries in Leaflet. Nice! I love open source software! I thought I’d mention the MySQL approach here to help expose Fabien’s solution to English searches and drive a little more traffic to his site. If you can’t read French, Google Translate is your friend! [...]
Friday June 06, 2014:
Convert Google Maps Polygon (API V3) to Well Known Text (WKT) Geometry Expression
There’s dozens of reasons why you might want the Well Known Text (WKT) geometry expression for a Google Maps Polygon object. Assuming you’re using the Google Maps API V3, and you’ve got a variable referencing your Polygon, I’ll suggest two approaches you can take to iterate over the paths and vertices in your Google Maps polygon and return the geometry expression as a Well Known Text string. Add a Simple Utility Method to Your Project Easy enough. Just add the following method to your project. Look below the method for an example of how you’d call it.
function GMapPolygonToWKT(poly)
{
// Start the Polygon Well Known Text (WKT) expression [...]
Monday January 27, 2014:
PostGREsql/PostGIS Implementation of Google’s Encoded Polyline Algorithm
[Edit 30 Jan, 2014] I added an additional PostGREsql method to perform Polygon encoding by concatenating polygon geometries (delimiter: †) and their inner rings (delimiter: ‡) together into one massive encoded block of ring features. I also provided an example JavaScript method demonstrating how to bring the amalgamated polygon feature encodings into your Google Map. By some uncanny twist of the fates, I’ve elected to use, had to use, and/or been asked to develop applications that use Google Maps ASCII Encoded Polyline expressions. In previous encounters, I’ve used a PHP class to handle the encoding task, and most recently I wrote a [...]
Friday November 08, 2013:
PostGIS: query all multipolygon parcels with at least one hole
I was writing some code to iterate over Well Known Text expressions for polygon features, and I decided I needed to test the most complex edge-case I could think of–multipolygon geometries where at least one of the bound polygons has a hole (i.e. an interior ring). I ended up with the following query. This seems like the kind of thing I’ll want to reuse later, so I’m noting it here. For good measure, I also use a rudimentary technique to sort the output with the most complicated geometries in the table at the top of the list. Basically, the more “text” it takes to describe the geometry using Well Known Text, the larger and more complex I figure it must be! [...]
Monday August 05, 2013:
Install httplib2 to your Preferred Python Runtime after ArcGIS Unceremoniously Hijacks the First Attempt
We’re going to use Google Maps Engine for some stuff, and so I thought I’d throw my first codes at it using Python. Well.. Google’s Python example requires the httplib2 library, so I needed to install that. When I did—following the library’s download/instructions—for some reason, the library went into service against ArcGIS’s embedded Python runtime, rather than my favored 2.7 instance, which I use for every single thing. ::grimaces:: To fix this, I just ensured the new library’s setup.py script was called from my preferred Python runtime rather than let Windows, or some black magic, decide which instance it should integrate with httplib2. [...]
Friday December 28, 2012:
osm2pgsql help and usage
I wanted a more convenient place to read the osm2pgsql help output, so this seemed as good a place as any for it. /E
-----------------------------------------------------------------
osm2pgsql -h osm2pgsql SVN version 0.80.0 (32bit id space)
Usage:
osm2pgsql [options] planet.osm
osm2pgsql [options] planet.osm.{gz,bz2}
osm2pgsql [options] file1.osm file2.osm file3.osm
This will import the data from the OSM file(s) into a PostgreSQL database suitable for use by the Mapnik renderer
Options: -a|--append Add the OSM file into the database without removing existing data. -b|--bbox Apply a bounding box filter on the imported data Must be specified as: minlon,minlat,maxlon,maxlat e.g. --bbox -0.5,51.25,0.5,51.75 -c|--create Remove existing data from the database. This is the default if --append is not specified. -d|--database The name of the PostgreSQL database to connect to (default: gis). -i|--tablespace-i [...]
Friday December 21, 2012:
Resident “lunatic” of LinkedIn
Just updated my LinkedIn profile.. This is just TOO FUNNY right now! [image] I totally had to save this for posterity. Unfortunately it will probably disappear before you can say “Jack Robinson”. 8)
Tuesday December 18, 2012:
mapnik rundemo.exe error: run from within the demo/c++ folder?
So I just installed the mapnik 2.0.1 binaries for Windows, and I ran into a “gotcha.” I couldn’t figure this out by Googling, so hopefully this post will help someone. (Be sure to note the embarrassing conclusion.) Specifically, following a fresh mapnik install, rundemo.exe told me this:
C:\mapnik-2.0.1rc0\demo\c++>rundemo.exe usage: ./rundemoNote the first bit: usage: ./rundemoUsually /usr/local/lib/mapnik Warning: ./rundemo looks for data in ../data/,Therefore must be run from within the demo/c++ folder.
Tuesday December 18, 2012:
osm2pgsql and windows errors: failed to start MSVCR90.dll, Connection to database failed, etc..
I’m following the BostonGIS tutorial(s) to learn how to setup an OpenStreetMap tile server on Windows (XP 32, cos’ that’s what’s on the desk), and I’m running into headache after headache. So this post notes the gotcha’s I’m encountering and how I’m fixing them (optimistically assuming I fix all of them). “Thar be dragons..” 8) ————————————————— Part I: “Failed to start MSVCR90.dll” First, as recommended I tried using the HOTOSM installer, but when I launch it using the example command provided on the BostonGIS site, I got the error above. I’d logged out/in, rebooted, ad nauseam, to no avail. Also the problem can’t be my redist libs–this install of XP is only a couple months old, and I have all the redists libs I know of (2005, 2008, 2010—plus, when I try reinstalling a redist it says I have the latest). I consider uninstalling and reinstalling Visual Studio to be more trouble than just trying something else to get the OSM download into PostGIS, so.. [...]
Saturday October 20, 2012:
Decode Google Map encoded points as Well Known Text (WKT) with Python
I had close encounter of the 5th kind yesterday.. here’s the gist.. It started when someone “gave” me a GIS dataset (..of polygons, kind of..) that a colleague of theirs, way back in ancient history, chose to pre-cook as ASCII-encoded point pairs. Their intention was almost certainly to use the pre-cooked data in Google Maps. Anyway, being arguably sane, I wanted to return this data to a more GIS-normal format so I could put it in a database like MySQL or Post and use it for other stuff. I considered a few different approaches to this problem, including creating a Google Map that could load-in all of the polygons from their encodings, then iterate over the polygons, interrogate the polygon point pairs, and finally concatenate WKT features from the points and save the geofeatures into a MySQL table. This approach offered the advantage of using Google’s existing Maps API to do the decoding for me. But let’s face it, that’s lame, uninspired, and not inventive.. it wasn’t even interesting. [...]
Saturday January 21, 2012:
PostGIS: count all features of each GeometryType in a spatial table
Sometimes, just when you think you’ve got something figured out –you get reminded that you really don’t. :/ As you may know, ESRI allows for single and muli-part geometries to live in the same FeatureClass. So, if I have a shapefile of roads, there might be both LINESTRING and MULTILINESTRING features in that dataset. I live just loose-enough not to care about that. But I do need to be aware of it when I’m cobbling data in PostGIS. In thise case, I was getting a PostGIS error tying to do a Dissolve-By-SQL, so I thought why not get a quick count of each GeometryType in the dataset? Maybe I was running into issues single and multi-part geometries were blurred together. It took me an embarassing chunk of time to get this right, so I figured I’d post the recipe in case I needed a reminder later. [...]
Saturday November 19, 2011:
Prepare a Shapefile for OpenScales using ogr2ogr and PostGREsql
This post explains how to import GIS data (a shapefile, in this case) into a database (PostGREsql) so it can be consumed by most any mapping API. I have OpenScales in mind, but this approach will support any mapping app with functions for rendering feature overlays using geodetic coordinates (i.e Longitude and Latitude). In many cases, you’ll need to translate your feature data out of a projected/cartesian system and into a geodetic/spherical system; so I’ll include a demonstration of that.
Quick and Dirty Summary
This approach has two parts. First, we’ll use GDAL’s ogr2ogr utility to import a shapefile into our database. Second, we’ll use a few SQL commands to translate our data from a projected to a geodetic system, as well as optimize the table for fast query speeds. [...]Friday November 18, 2011:
ogr2ogr: Export Well Known Text (WKT) for one feature to a CSV file
Perhaps you’re looking for this?
ogr2ogr -f “CSV” “E:\4_GIS\NorthArkCartoData\UnitedStates\MO_wkt” “E:\4_GIS\NorthArkCartoData\UnitedStates\USStates.shp” -sql ” SELECT * FROM usstates WHERE STATE_NAME = ‘Missouri’ ” -lco “GEOMETRY=AS_WKT ” -lco “LINEFORMAT=CRLF” -lco “SEPARATOR=SEMICOLON”My buddy at work needed a way to get the WKT geometry definition for a single feature in a shapefile. I thought, “surely this is something we can do with OGR?” Lo’ and behold, yes it was. [image] The script above uses OGR SQL to interrogate a shapefile for one lone feature and, when it’s found, exports the record to a comma separated values (CSV) file (or in this case, a semicolon delimited file). Here’s a quick break down: [...]
Sunday November 06, 2011:
OpenScales: Coordinate Transformation from Uncommon Projections
———— UPDATE 2.19.2013: It turns out you don’t have to rebuild OpenScales to apply uncommon projections, you can just assign the projection directly to ProjProjection.defs. (See Simon L.’s answer here in the OpenScales user group.) So if I modify my example toward the bottom of the post (at #5), here is how I would apply the alternate, and arguably better, approach. (Fair warning, I haven’t compiled this and tested it yet, the only difference is the first line), but this is how I expect it should be. If it doesn’t work like this, it’s likely ‘cos an instance of ProjProjection needs to be created first, and the defs property of the instance needs to be appended. In other words, if this doesn’t work, follow the line instantiating moNADProj with moNADProj.defs[‘EPSG:102697’] = “+title…. blah blah”; One of these days I’ll make super-sure this is correct. But I didn’t want to forget where I found this, hence updating the post.) [...]
Saturday November 05, 2011:
Nikon Coolpix 3100 Near Infrared Hack
While this post is not in-theme with the stated goals of the blog, I wanted to save my notes somewhere as well as make a URL to show-off for my friends. On the other hand, I’m opimistic my experiment will enable some “eco-minded research”, so perhaps there’s still a loose tether linking this back to the GIS community.. Basically, I hacked my old, 3 megapixel Nikon Coolpix 3100 today so that I could take NIR (near infrared) pictures. The following images serve as a storyboard for the process. 0: The tools and materials involved include (clockwise from bottom-left) —-> small round-nose pliers (my wife uses these for beading) [...]
Monday October 17, 2011:
Install GDAL on Windows
Later posts on this blog will assume a working install of GDAL/OGR, so it’s prudent that I first demonstrate how to get a fresh cut of GDAL/OGR up-and-running on a Windows system. Windows users have a few options for installing GDAL (see this question at gis.stackexchange,which will point you to Christoph Gohlke’s binaries –scroll down to GDAL, and the OSGeo4W Installer, among other approaches). However, I’ll guide you through what works for me, which can be summarized as follows: 1) Install Python 2) Install the GDAL binaries published by Tamas Szekeres 3) Append your environment Path variable [...]
Sunday October 16, 2011:
Google Maps for Flash/Flex: Dead Man Walking
“When life gives you lemons..” Let’s just review the signs:
- First, the Google Maps Flash/Flex API had not been updated for quite some time.
- Meanwhile, some developers were sharing a hack to bypass a nasty initialization bottleneck in the Maps Flash/Flex API that stalled map loading in AIR apps.
- At the other end of the yard, Google lifted its registration key requirement for the Maps JavaScript API V3; yet, the Flash/Flex counterpart remained unchanged in this respect (curiously, it was key-hashing that caused AIR apps to stall).
- Finally, and though unrelated, Pamela Fox, one of the higher-profile support engineers attached to Maps developer relations, left Google for other pursuits. [...]
Answers at gis.stackexchange.com
A bug in ESRI Line Geometry WKB that adds needless, trailing 0's behind otherwise valid WKB?
My co-worker answered my main questions. Q1: Is the ESRI Line Geometry as WKB really invalid? [Yes.] The wkb says there is 9 points in the line, and then the parser should read 9 points. MS (rightfully) detects more data (being the 10 0's) and tells its messed up. The 10 0's are 5 bytes in the original array. Remove the last 10 0's, and its good, like.. [...]
My co-worker answered my main questions. Q1: Is the ESRI Line Geometry as WKB really invalid? [Yes.] The wkb says there is 9 points in the line, and then the parser should read 9 points. MS (rightfully) detects more data (being the 10 0's) and tells its messed up. The 10 0's are 5 bytes in the original array. Remove the last 10 0's, and its good, like.. [...]
How can I convert PostGIS geography to geometry and split polygons that cross the antimeridian
Pete's answer was helpful, although I was getting an error like,
Pete's answer was helpful, although I was getting an error like,
aggregate function calls cannot contain set-returning function calls. Nevertheless, I got it working using that strategy then took it a step further and turned it into a function for easy reuse. Hopefully someone finds this helpful:
-- Replace 'aaa' with whatever "namespace" you group your functions in.
CREATE OR REPLACE FUNCTION
public.aaa_split_linestring_at_meridian( _line_geom geometry )
RETURNS geometry
LANGUAGE plpgsql
AS $function$
declare
_split_line_geom geometry;
begin
WITH t AS (
SELECT
[...]How/Where can I find vector data for building footprints in Texas?
If I wanted some ready-to-go building footprints, I'd start calling county Assessor offices and city planning departments. I think they will be your best bet for ready-made building footprint polygons. However because these are expensive data, I believe it's rare to find them free for the taking. It's likely even rarer to find it associated with sparsely-populated areas. :/ [...]
If I wanted some ready-to-go building footprints, I'd start calling county Assessor offices and city planning departments. I think they will be your best bet for ready-made building footprint polygons. However because these are expensive data, I believe it's rare to find them free for the taking. It's likely even rarer to find it associated with sparsely-populated areas. :/ [...]
Bulk lookup of address census tract and block
Ok Ben, here are my assumptions:
Ok Ben, here are my assumptions:
- You've already got your data (I had some address points in a shapefile, and I downloaded census tract and census block shapefiles for Missouri).
- You've already geocoded your address points and you're comfortable projecting the data.
- You're comfortable with an OGR/PostGIS [...]
Dissolve or unsplit lines on common attributes in PostGIS or GRASS?
Interesting question. I don't consider myself a PostGIS guru, but I toyed with your problem and came up with the following query that dissolves highway
Interesting question. I don't consider myself a PostGIS guru, but I toyed with your problem and came up with the following query that dissolves highway
LINESTRING records into MULTILINESTRING records when they have common values in multiple fields (in my dataset, I matched on the name and state fields). I used OGR to push a highways shapefile (renamed to ushwys) into PostgreSQL; so my geometry field is called [...]Where can I find country borders data in plain text format? (XML, JSON, CSV, etc)
I like @celenius' answer; however, another option would be to export the the World Borders Dataset to CSV using ogr2ogr. I just downloaded your preferred dataset and ran this command with ogr2ogr --note that your script should be a single string without any line breaks. I find them easiest to write in notepad with wordwrap turned on, then I copy them into my ogr2ogr terminal: [...]
I like @celenius' answer; however, another option would be to export the the World Borders Dataset to CSV using ogr2ogr. I just downloaded your preferred dataset and ran this command with ogr2ogr --note that your script should be a single string without any line breaks. I find them easiest to write in notepad with wordwrap turned on, then I copy them into my ogr2ogr terminal: [...]
What is correct format for a WKT linestring file with date/time in QGIS
@Vic your date field is merely a separate data field and irrelevant to Well Known Text (WKT), which is just one of many standardized ways of representing feature geometry. The
@Vic your date field is merely a separate data field and irrelevant to Well Known Text (WKT), which is just one of many standardized ways of representing feature geometry. The
oid field in your example is merely a feature identifying unique id value. Depending where you look in the wild, you'll find these have many aliases: [...]
Adding field with filename when merging shapefiles with ogr2ogr
Here is how I accomplished this problem by using a Python script to daisy-chain several ogr2ogr instructions together. You could easily convert it to a batch script, basically I just concatenate together ogr2ogr instructions (
Here is how I accomplished this problem by using a Python script to daisy-chain several ogr2ogr instructions together. You could easily convert it to a batch script, basically I just concatenate together ogr2ogr instructions (
cmd), then execute them by calling os.system(cmd), passing-in the ogr2ogr command I concatenated together. [...]Attempting to use ogr2ogr to convert shapefile to SQL Server table; getting error "unable to find driver mssqlspatial"
See my answer here for a little more detail, but what happens if you add this to your
See my answer here for a little more detail, but what happens if you add this to your
ogr2ogr command?
--config MSSQLSPATIAL_USE_BCP NO
ogrinfo doesn't respond while exporting to MSSQL with ogr2ogr
@Goran, I've had mixed experiences using ogr2ogr to import into MSSQL. Out of curiosity, you might try adding this to your command just to see what happens:
@Goran, I've had mixed experiences using ogr2ogr to import into MSSQL. Out of curiosity, you might try adding this to your command just to see what happens:
--config MSSQLSPATIAL_USE_BCP NO
See this answer and the question it's posted on for a little more detail. Of noteâI recommend trying this option even if you don't think it's relevant! [...]
Searching for c# code to convert from UTM to WGS1984 (and back)
if you're willing to use the GDAL/OGR C# bindings, which you can get here assuming you use Windows, you can achieve what you want via the
if you're willing to use the GDAL/OGR C# bindings, which you can get here assuming you use Windows, you can achieve what you want via the
OSGeo.OSR library and proj4 projection expressions from spatialreference.org.
The specifics of finding and installing GDAL/OGR builds for your operating system are covered in other questions and several blog posts. But once you find a build that suits you, make sure you reference the [...]
Bentley DGN format to Oracle Spatial
@Rao, ogr2ogr can do this (at least, the OGR formats page shows both Microstation DGN as well as Oracle Spatial); however, you'll need a GDAL/OGR installation that includes support for Oracle. If you're using Windows, you should find everything you need on Tamas Szekeres site, otherwise check here and look for an OS-specific approach to installing GDAL. [...]
@Rao, ogr2ogr can do this (at least, the OGR formats page shows both Microstation DGN as well as Oracle Spatial); however, you'll need a GDAL/OGR installation that includes support for Oracle. If you're using Windows, you should find everything you need on Tamas Szekeres site, otherwise check here and look for an OS-specific approach to installing GDAL. [...]
ogr2ogr SHP to MSSQL import leaves geometry column empty
Similar symptoms, but I had to add
Similar symptoms, but I had to add
--config MSSQLSPATIAL_USE_BCP NO to my ogr2ogr instruction. I think a consequence of this parameter is that the ogr2ogr process runs slower than it could under ideal circumstances.
Of note: I installed GDAL using the OSGEO installer, which for some reason didn't include anything in the [...]
Seeking library for computing accurate geographic buffer around coordinate in EPSG:4326?
C#.Net Solution using GDAL/OGR bindings..
If you're willing to add the GDAL/OGR C# bindings to your .Net project (see second heading), then you can do what you want within a .Net environment. It's the same thing I did above in my prior answer using JavaScript, but 100% in C# here, and only one libraryâ the GDAL C# bindings. [...]Seeking library for computing accurate geographic buffer around coordinate in EPSG:4326?
You might consider 1: defining an Azimuthal Equidistant projection centered on the point you want to buffer, 2: reprojecting your coordinate from LatLong to the custom azimuthal projection, then 3: buffering the point using whatever library you prefer against the Azimuthal Equidistant coordinates, which preserves distance so should give you a clean buffer, then 4: iterate back over the new coordinates an [...]
You might consider 1: defining an Azimuthal Equidistant projection centered on the point you want to buffer, 2: reprojecting your coordinate from LatLong to the custom azimuthal projection, then 3: buffering the point using whatever library you prefer against the Azimuthal Equidistant coordinates, which preserves distance so should give you a clean buffer, then 4: iterate back over the new coordinates an [...]
Does Analyze Actually Do Anything?
An anecdotal response here: My first foray into using Open Source GIS tools and PostGRES/PostGIS was around PostGIS 7.x on a very small instance ..I was trying to make a little web mapping app that used PHP to spatially query features from PostGIS, then render the feature vectors in the map so that [...]
An anecdotal response here: My first foray into using Open Source GIS tools and PostGRES/PostGIS was around PostGIS 7.x on a very small instance ..I was trying to make a little web mapping app that used PHP to spatially query features from PostGIS, then render the feature vectors in the map so that [...]
DLL error in various GDAL installations
OP, ok so I'm the author of the cartometric.com blog resource you followed that left you with this issue. :) Turns out, I just retraced my steps that process while installing a fresh copy of GDAL on my work desktop (Win 7/64 bit, but I use 32-bit Python and GDAL), and I encountered the same problem. [...]
OP, ok so I'm the author of the cartometric.com blog resource you followed that left you with this issue. :) Turns out, I just retraced my steps that process while installing a fresh copy of GDAL on my work desktop (Win 7/64 bit, but I use 32-bit Python and GDAL), and I encountered the same problem. [...]
How would I process and output every possible route of a stream network?
Not a real answer, but more than a comment.. TauDEM should be able to do what you want, and QGIS provides plugin support for it. There is an existing question here on GIS.SE that you might want to review. [Update/Response to comment] Ultimately your workflow will need to utilize several of the methods [...]
Not a real answer, but more than a comment.. TauDEM should be able to do what you want, and QGIS provides plugin support for it. There is an existing question here on GIS.SE that you might want to review. [Update/Response to comment] Ultimately your workflow will need to utilize several of the methods [...]
TauDEM fails to create stream network shapefile (-net option) when running StreamNet method on Ubuntu
It turns out there is a minor bug in the create shapefile implementation, or depending on how you want to classify it, it could also just be an issue with the StreamNet method's documentation. When reviewing the third link I cited in the original question, I noticed where one user described a solution.. [...]
It turns out there is a minor bug in the create shapefile implementation, or depending on how you want to classify it, it could also just be an issue with the StreamNet method's documentation. When reviewing the third link I cited in the original question, I noticed where one user described a solution.. [...]
How to display shapefiles on an OpenLayers web mapping application that are provided by users as an input without making any changes in the database
From a (very) high level, for users to upload their own shapefiles and have your server manage the display/tiling/interactivity, as if it's any other layer, you will need to..
From a (very) high level, for users to upload their own shapefiles and have your server manage the display/tiling/interactivity, as if it's any other layer, you will need to..
- Create a dialog in your UI where they can select a file from their system to upload to your server.
- Write a web service of some sort that.. a) receives the uploaded file b) stores it somewhere appropriate on your server [...]
Converting CSV file of WKT data to shapefile using ogr2ogr?
On Linux/Ubuntu, I needed to do something similar today and came up with the following solution. It uses a few helper utilities that were already in my Ubuntu shell, but it's an effective one-liner that doesn't require creating any VRT sidecar, etc. It's a handy way to solve "I have some WKT feature [...]
On Linux/Ubuntu, I needed to do something similar today and came up with the following solution. It uses a few helper utilities that were already in my Ubuntu shell, but it's an effective one-liner that doesn't require creating any VRT sidecar, etc. It's a handy way to solve "I have some WKT feature [...]
Reprojecting point if I only have the source TIFF extent
Reading back over your question, I think your proposal to reproject each point while creating the SHP using pySHP is the most reasonable considering the path you've already started down. (Although, as stated in the comments, this is not how I would have approached such a project..) To do this, you'll need a system to keep your classifier/shapefile-writer oriented relative to the original TIFF's X/Y values as it moves through your JPEG tiles. [...]
Reading back over your question, I think your proposal to reproject each point while creating the SHP using pySHP is the most reasonable considering the path you've already started down. (Although, as stated in the comments, this is not how I would have approached such a project..) To do this, you'll need a system to keep your classifier/shapefile-writer oriented relative to the original TIFF's X/Y values as it moves through your JPEG tiles. [...]
How to deal with several markers in the same location on a map, e.g. to let viewers know they're there, and regarding tooltips with different info
This isn't your exact technology, but you might want to look through the code for some of Leaflet's Clustering/Decluttering plugins. If I recall, we use the Leaflet.markercluster plugin in our stuff. It looks kinda like this..
Then, if you click on one of the clustering icons (I chose a VERY busy one for example), notice how it offsets the individual icons that would otherwise be overlapping using a spiral sequence? I'm pretty sure they just take the icon's original coordinates (that is, screen or div coordinates) then ad [...]
This isn't your exact technology, but you might want to look through the code for some of Leaflet's Clustering/Decluttering plugins. If I recall, we use the Leaflet.markercluster plugin in our stuff. It looks kinda like this..
Then, if you click on one of the clustering icons (I chose a VERY busy one for example), notice how it offsets the individual icons that would otherwise be overlapping using a spiral sequence? I'm pretty sure they just take the icon's original coordinates (that is, screen or div coordinates) then ad [...]
Simple Asset Web Map Service
I recommend you consider Leaflet or OpenLayers ..personally, I prefer Leaflet. If you don't have "that many" location points, I'd just convert them into a JSON file or a GeoJSON file and load them all on $(document).ready(). When I say "that many", obviously that threshold is subjective. I consider it to be around 2000 points/features. [...]
I recommend you consider Leaflet or OpenLayers ..personally, I prefer Leaflet. If you don't have "that many" location points, I'd just convert them into a JSON file or a GeoJSON file and load them all on $(document).ready(). When I say "that many", obviously that threshold is subjective. I consider it to be around 2000 points/features. [...]
Why the replace function in PostgreSQL is not returning the value?
OP based on your image, your
OP based on your image, your
replace() function is trying to match something like..
Puerto Rico ..which is the value in the database, with..
(Puerto Rico) ..which is the value you're telling it to match in the second argument of your replace() function.
Remove the ( and the ) from the second argument in your [...]
How to order points along a line?
If using PostGIS, you can order the bus stops according to their value returned for an
This is how it looks if I apply it to the linestring and points you supplied:
If using PostGIS, you can order the bus stops according to their value returned for an
ST_Line_Locate_point() call.
This is how it looks if I apply it to the linestring and points you supplied:
SELECT
ST_Line_Locate_Point(line.f1, ST_GeomFromText('POINT(-70.58176091969386334 -33.37599871220383818)')) as A,
ST_Line_Locate_Point(line.f1, ST_GeomFromText('POINT(-70.58081166577235877 -33.37588970535671251)')) as B,
ST_Line_Locate_Point(line.f1, ST_GeomFromText('POINT(-70.57890070625846 [...]Ubuntu: Updating Mapnik from 2.2.0 to 3.x, complications?
I remembered I had a tiny Rackspace instance that I setup once upon a time to test code portability. It has a nearly identical version of our webstack, particularly the Ubuntu and Mapnik versions, so it made a reasonable test bed to experiment with moving from 2.2.x to 3.x. In a nutshell, this was a complicated transition that involved sincere troubleshooting, research, stack resets, some code modifications to XML stylesheets, and although I got things working again, my confidence is a little weak. Were this a production system, downtime would have been considerable. [...]
I remembered I had a tiny Rackspace instance that I setup once upon a time to test code portability. It has a nearly identical version of our webstack, particularly the Ubuntu and Mapnik versions, so it made a reasonable test bed to experiment with moving from 2.2.x to 3.x. In a nutshell, this was a complicated transition that involved sincere troubleshooting, research, stack resets, some code modifications to XML stylesheets, and although I got things working again, my confidence is a little weak. Were this a production system, downtime would have been considerable. [...]
Renderd outputs "failed to initialize projection 4326"
Your issue may have something to do with projection definitions defined in the stylesheet using XML entities. I stumbled across your question while troubleshooting a cascade of issues encountered in an attempt to upgrade from Mapnik 2.2 to 3.x, as inquired upon here. Anyway in my quest, I eventually got Apache errors like this: [...]
Your issue may have something to do with projection definitions defined in the stylesheet using XML entities. I stumbled across your question while troubleshooting a cascade of issues encountered in an attempt to upgrade from Mapnik 2.2 to 3.x, as inquired upon here. Anyway in my quest, I eventually got Apache errors like this: [...]
Density Mapping with mapnik
Way late to the party here, but if I were in your situation, I'd create a new table in the database and write a script to iterate over your existing points, and for each point, create N representative records corresponding to your total_count property for that parent-record. If you want to take it a small step farther, for each child-record, I'd offset its lat/lon values a statistically-insignificant distance on both lat and lon so any software that touches it (i.e. Mapnik, etc.) will see a truly unique point. I'm not sure how granular or "zoomed-in" your spatial data i [...]
Way late to the party here, but if I were in your situation, I'd create a new table in the database and write a script to iterate over your existing points, and for each point, create N representative records corresponding to your total_count property for that parent-record. If you want to take it a small step farther, for each child-record, I'd offset its lat/lon values a statistically-insignificant distance on both lat and lon so any software that touches it (i.e. Mapnik, etc.) will see a truly unique point. I'm not sure how granular or "zoomed-in" your spatial data i [...]
Citing QGIS in formal publications
Start here: https://qgis.org/en/site/getinvolved/faq/index.html#how-to-cite-qgis
Start here: https://qgis.org/en/site/getinvolved/faq/index.html#how-to-cite-qgis
Q: How to cite QGIS? A: To cite QGIS software in your piece of work, for work or an assignment, this general citation might be helpful: "QGIS Development Team (YEAR). QGIS Geographic Information System. Open Source Geospatial Foundation Project. [...]