Automation Machine

Proof of Automation

Only editable by group admins

  • Last updated December 14, 2017 at 12:42 PM
  • Evidence visible to public
Write up a summary about how you were able to change the process, and how the current automation process works.

All posted evidence

Automated dataset pull for OMB and gave them a great looking interface to search through the data.

Some of this is private so I will be very vague with the explanation. OMB needed a way to look through a dataset... The problem is that the dataset was being provided as a TXT file in an FTP server, so they would have to manually connect via FTP and download the file, and open it up with excel. I created a cron job that runs a bash script that I wrote from scratch to connect to the FTP, download the newest file, save it to a particular directory, and then do a CURL request to a particular website that then consumes the TXT file, parses it and inserts it into a database. OMB is then able to log in to the website that I created for this and search through the data in order to perform one of their daily tasks. Also, this web application will also be used by Codes and Regulations :). If anyone working for Louisville Metro would like to see this web application, as well as the cron job, contact me for a demo.
jorgefelico Over 6 years ago

Automated uploading of datasets to the open data portal, and subsequent ingestion into the catalog.

I created a text file of urls that is processed by cron on data.louisvilleky.gov, that loops through each file url, downloads and imports it into the Open Data catalog. From there, it uses drush to automatically import a select group of datasets into MySQL, which also makes those datasets available via a REST API.
mattgolsen Over 6 years ago