-
-
Export Airtable Grids to CSV
I have to say that when I first saw Airtable I thought, oh look – Excel online. It’s a little more than that and it’s been kinda’ cool to watch Airtable mature. The thing I like about Airtable is that a lot of software – in fact most software, is a database, logic that puts data into the database and takes things out of the database, and then actions based on the data. If we’re interacting with operating system APIs, then that database might be in registry hives (Windows) or defaults domains (Mac). We like to overcomplicate what software does in our brains, but that’s the gist. And a lot…
-
Easy Python Module Version Management
Python comes with a handy-dandy little tool called pip for installing modules that you import in scripts. Python3 comes with a hand-dandy little tool called pip3 for installing modules you import in scripts. A practice I try to follow is to take the version of a module I was using when I wrote a script and create a requirements.txt file alongside it. Basically, if I import then I’ll add what I imported into the requirements.txt file. This would usually follow the below syntax: tensorboard==2.0.2 tensorflow==2.0.0 tensorflow-estimator-2.0.1 werkzeug-0.16.0 wrapt-1.11.2 Now that I have a requirements file, I can use that to install packages on the next host I run the script…
-
Lightweight Python Tagger Using NLTK
Recently I was looking at automating the tagging of content from a mysql database. NLTK (Natural Language Training Kit) to the rescue to make this pretty easy. This GitHub project wraps NLTK into a small script to tag and provides a class for excluding words: Once the tagger was written we could add cursor.execute() or cursor.fetchall() as a way to bring in input from mysql content or since it’s a one-time run, we can use a simple mysql query run from bash and dump the output into the script as follows: mysql -X -u $supersecretUSER -p$supersecretPASS -h$HostName -D$MyDB -e'SELECT notes FROM tablename > notes.json lightweighttagger.py In the above just swap…
-
7 Python Machine Learning Modules To Get Acquainted With
A lot of machine learning work has been done in python. Therefore, there are a few python modules that make the work of normalizing data, analyzing data, and training algorithms much easier. Let’s look at a few of these and if you have suggestions, please feel free to comment! Tensorflow Tensorflow is more of an ecosystem, but you can import it into a python project and build some pretty amazing stuff relatively quickly. You usually see it imported as just tf: While it helps to have a basis in linear algebra and vector calculus that’s not entirely necessary. A tensor is an object that is similar to a vector but…
-
Linear Algebra in Python Scripts
Preparing data for tensorflow is often easiest when done as a two step process. In machine learning, you often get into trying to plot points, calculate tangents, and a lot of basic algebra. Working out equations kinda’ reminds me of being in in-school suspension in high school. Except now we’re writing code to solve the problems rather than solving them ourselves. I never liked solving for a matrix… But NumPy is a great little framework to import that does a lot of N-dimensional array work. A few basic tasks in the following script includes a number of functions across norms, matrix products, vector products, decompose, and eigenvalues. Remove/comment what you…
-
Create Quick And Dirty PDFs from Web Pages in Python
Let’s say you want to make a script that creates a PDF of a web page. pdfkit makes that pretty easy. Simply import pdfkit and then call pdfkit.from_url, passing along the source location as your first parameter and the resultant file as your second, as follows, using http://docs.jamf.com/10.10.1/jamf-pro/release-notes/What’s_New_in_This_Release.html as our source and just calling the pdf we create Release_Notes.pdf: import pdfkit pdfkit.from_url('http://docs.jamf.com/10.10.1/jamf-pro/release-notes/What's_New_in_This_Release.html', 'Release_Notes.pdf') Your source location could also be from a standard html file (e.g. if you’re running from your site location) and for those you’d use pdfkit.from_file instead of pdfkit.from_url. If you don’t have pdfkit installed, you might need to pip it first: pip install pdfkit One last note,…
-
Python Script To Move A SQL Database To .csv Files
You have a database, such as a mysql dump of a Jamf Pro server, or a sql dump of a WordPress site. You want to bring it into another tool or clean the data using a csv as an intermediary. Or you’re using an Amazon Glue job to ETL the data. The following script will take that sql dump and convert it into a bunch of csv files. To use the script, simply run it with $1 as the path to the sql file and $2 as the path to the export directory, as follows: python sqlcsvexport.py /sql_file_path /target_dir To download the script:
-
A Bit About Python Module Imports
The first lines of practically every python script ever begin with the word import. Per the python docs at https://docs.python.org/2/tutorial/modules.html a module is “a file containing Python definitions and statements.” To open a python environment, simply open Terminal and type Python python To import a module just type import followed by the name of the module. A common one is os, so let’s import that: import os Now let’s use str to see where it is: str(os) You’ll then see what the script was that is actually the os module and where it can be found/edited/viewed. Now let’s do a couple of basic tasks with os. First, let’s grab the…
-
Define docstrings in Python
Bryson mentioned Docstrings in the latest episode of the MacAdmins Podcast. But how do you use them? Documentation strings (or docstrings for short) are an easy way to document Python objects (classes, functions, methods, and modules in-line with your code. Docstrings are created using three double-quotes in the first statement of the definition and are meant to describe in a human readable way what the object does. Let’s look at an example for hello_krypted: def hello_krypted(): """This simply echos Hello Krypted. But there's so much potential to do more! """ Docstrings can then be accessed using the __doc__ attribute on objects (e.g. via print): >>> print hello_krypted.__doc__ This simply echos…