Scripting Video Changes in Lion: avconvert, pcastaction & podcast

The avconvert command is a new addition in OS X Lion that allows administrators to quickly and easily convert video from one format to another using presets for video conversion. The presets are mostly common Apple formats tuned to specific devices. In its simplest form, avconvert uses a preset, a source and then an output to convert the source to the output using the preset to define the format to use for conversion. A useful preset is the 640×480 one. To convert this using this idea: /usr/bin/avconvert --preset Preset640x480 --source /Convert/ --output Converted/ While some of the presets are pretty self explanatory, I haven’t gone through them all to see their specific outputs. Simply regressioning through each and then doing a get-info on the resultant QuickTime should net such a result. You can also use avconvert to extract audio or video only, to change data rates, track height and width, convert codecs, change frame rates and event o frame reordering. You can also specify a closed caption track, thumbnail tracks and all kinds of other cool stuff. While avconvert is the latest addition to video augmentation commands, the pcastaction command has also received some new features. I had previously written up a list of verbs for pcastaction at This list is further enhanced in Lion. New verbs include:
  • addchapter – adds a chapter at a specified time
  • addtracks – add tracks using an optional offset and layers
  • deletetracks – deletes audio, video or audio_and_video tracks
  • extracttracks – outputs audio, video or audio_and_video tracks
  • flatten – flattens .mov, .m4v, .m4a and .m4b movies
  • join – joins two input files into an output file with an optional gap
  • qtimport – prepares QuickTime files with optional chapterizing
  • qtinfo – obtains keys from QuickTime files
  • sharewithpodcastlibrary – option for submitting a file to another Podcast Producer library
  • split – splits a QuickTime movie and outputs two files that are split at the time specified in the –time option
  • trim – specify start and end and remove the rest of the file
The podcast command, used to run Podcast Producer workflows is also still around and can be very useful. While there isn’t new stuff, it is worth mentioning that –addacl becomes –addaccess, –make explicit becomes –makefeedexplicit and –makenonexplicit becomes –makefeednonexplicit. There’s also the long, long awaited option for –removeepisode. This last option allows administrators to remove episodes from the Podcast Producer library based on the UUID of the episode. In my testing, you still need to remove the entry from the blog if you are also exporting episodes to the a blog, but this is basically what we were doing in the Deleting a Podcast post I did some time ago at, just they wrap the three commands into one option of the podcast command. Still look to the asset removal article I did for actually scrubbing files ( . Feed removal is also still manual: ACL management remains about the same ( While I never got around to writing up how to programatically manage ACLs in Podcast Producer, it is worth mentioning that podcast’s –addacl option and –enableacl will allow you to do so. They, along with other options in podcast and pcastaction are much better documented in the man pages in lion, so the things I couldn’t get to work in 10.6 should be sorted out somewhere in 10.7. Finally, while nothing new, the work I did on image file automation ( with sips I now have hooking into podcast workflows. I hope to publish an article on this at some point in the future, but the idea is a workflow where you drop an image and a video into a folder (or use an upload dialog) and it watermarks the video with a compressed-down version of the image… Also, avconvert offers a perfect compliment to podcast workflows. I’ve had a number of instances where people were trying to feed pcastaction formats that were unsupported, video that was too large or other problematic inputs and so avconvert allows us to sanitize the inputs for pcastaction or podcast prior to managing our workflows. With launchdaemons watching directories this provides some of what Final Cut Server was able to provide, only without the database of assets, easy way to tag them, etc, etc, etc, etc. Overall, a very nice incremental update to Podcast Producer and en masse video management in Lion. Nothing jaw dropping or massive, but some nice new features, better documentation and in my testing so far, more overall stability.

Managing Mail and Safari RSS Subscriptions from the Command Line

Safari can subscribe to RSS feeds; so can Mail. Podcast Producer is an RSS or XML feed as are the feeds created by blog and wiki services in Mac OS X Server. And then of course, RSS and ATOM come pre-installed with practically every blogging and wiki tool on the market. Those doing mass deployment and scripting work can make use of automatically connecting users to and caching information found in these RSS feeds. If you have 40,000 students, or even 250 employees, it is easier to send a script to those computers than to open the Mail or Safari client on each and subscribe to an RSS feed. Additionally, pubsub offers what I like to call Yet Another Scripting Interface to RSS (my acronym here is meant to sound a bit like yessir).  Pubsub caches the feeds both within the SQLite database and in the form of XML files. Because pubsub caches data onto the client it can be parsed more quickly than using other tools, allowing a single system to do much more than if a feed were being accessed over the Internet. Using pubsub We’ll start by looking at some simple RSS management from the command line to aid in a quest at better understanding of the underpinnings of Mac OS X’s built-in RSS functionalities. The PubSub framework stores feeds and associated content in a SQLite database. Interacting with the database directly can be a bit burdensome. The easiest way to manage RSS from Mac OS X is using a command called pubsub. First off, let’s take a look at all of the RSS feeds that the current user is subscribed to by opening terminal and simply typing pubsub followed by the list verb: pubsub list You should then see output of the title and url of each RSS feed that mail and safari are subscribed to. You’ll also see how long each article is kept in the expiry option and the interval with which the applications check for further updates in the refresh option. You can also see each application that can be managed with pubsub by running the same command with clients appended to the end of it (clients are how pubsub refers to applications whose subscriptions it can manage): pubsub list clients To then just look at only feeds in Safari: pubsub list client And Mail: pubsub list client Each of the above commands will provide a URL for the feed. This url can be used to show each entry, or article in the feed. Extract the URL and then you can use the list verb to see each feed entry, which Apple consistently calls episodes both within PubSub, in databases and on the Podcast Producer server side of things but yet somehow calls an entry here (consistency people). To see a list of entries for a given URL: pubsub list Episodes will be listed in 40 character hex keys, similar to other ID space mechanisms used by Apple. To then see each episode, or entry, use the list verb, followed by entry and then that key: pubsub list entry 5fcef167d77c8c00d7ff041a869d45445cc4ae42 To subscribe to a pubsub, use the –client option to identify which application to subscribe in along with the subscribe verb, followed by the URL of the feed: pubsub --client subscribe To unsubscribe, simply use pubsub followed by the unsubscribe verb and then the url of the feed: pubsub unsubscribe Ofline Databases and Imaging While these can be run against a typical running system, they cannot be run against a sqlite database that is sitting in all of your users home folders nor can they be run against a database in a user template home on a client. Therefore, to facilitate imaging, you can run sqlite3 commands against  database directly. The database, stored in ~/Library/PubSub/Database/Database.sqlite3. To see the clients (the equivalent of `pubsub list clients`): sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'SELECT * FROM clients' To see each feed: sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'SELECT * FROM feeds' To see each entry: sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'SELECT * FROM entries' To see the column headers for each: sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(Clients)'; sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(Feeds)'; sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(Subscriptions)'; sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(Entries)'; sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(Enclosures)'; sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(Authors)'; sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(Contents)';     sqlite3 /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'PRAGMA TABLE_INFO(SyncInfo)'; To narrow an ID down to a specific row within any of these searches add a WHERE followed by the column within the table you’d like to search. For example, if we wanted to only see the article with the identifier of 5b84e609317fb3fb77011c2d26efd26a337d5d7d sqlite3 --line /Volumes/Image/Username/Library/PubSub/Database/Database.sqlite3 'SELECT * FROM entries WHERE identifier="5b84e609317fb3fb77011c2d26efd26a337d5d7d"' Note: Sqlite3 can use the –line option to show each entry in an XML feed per line. Dumping pubsub to be Parsed By Other Tools Pubsub can also be used as a tool to supply feeds and parse them. You can extract conversations only matching specific patterns and text or email yourself that they occurred without a lot of fanfare. You can also dump the entire feed’s cached data by specifying the dump verb without the entry or identifier but instead the URL: pubsub dump Once dumped you can parse the XML into other tools easily. Or to dump specific entries to XML for parsing by another tool using syntax similar to the list entry syntax: pubsub dump entry 5fcef167d77c8c00d7ff041a869d45445cc4ae42 Because these feeds have already been cached on the local client and because some require authentication and other expensive (in terms of script run-time) processes to aggregate or search, looking at the files is an alternative way of doing so. Instant refreshes can also be performed using pubsub’s refresh verb followed by a URL: pubsub refresh Also, feeds are cached to ~/Library/PubSub/Feeds, where they are nested within a folder with the name of the unique ID of the feed (row 2 represents the unique ID whereas row 1 represents the row). Each episode, or post can then be read by entry ID. Yhose entries are basic xml files. You can also still programatically interface with RSS using curl. For example: curl --silent "http://${server}" | grep "item rdf:about=" | cut -c 18-100 | sed -e "s/"//g" | sed -e "s/>//g"

Deleting a Podcast in Podcast Producer

In an earlier post I looked at querying feeds and removing objects from Podcast Producer in a somewhat broad manner. To delete a single podcast (not a feed), you need to first find the ID for the podcast, then delete the corresponding podcast bundle (pdb). Removing assets is one of the only processes with regards to Podcast Producer that isn’t intuitive, and it’s just waiting for someone to wrap these steps into a nice pretty GUI… To find the ID of the podcast, first let’s query the sql database for the title: sqlite3 /Volumes/pcp/PodcastProducer/Server/db.sqlite3 'SELECT * FROM episodes' | grep "TITLE" Note the date and the ID (the big long string) and then browse to /Volumes/pcp/PodcastProducer/Content and then the folder that corresponds to the date obtained from the sqlite3 command. Once opened find the ID (or “big long string”) from the sqlite3 command. Then remove the Podcast Data Bundle that corresponds to that ID: rm -rf .pdb Once the entry has been deleted then resynchronizing the library will get rid of it completely. This can be done using the pcastconfig command along with the –sync_library option: pcastconfig --sync_library

Scrubbing Assets from Podcast Producer

At some point, you may find that you would like to remove all episodes from Podcast Producer that were brought in using a specific workflow, or based on a specific keyword, a string in the title, a date, or the user that created the episodes. All of these attributes are trapped in the db.sqlite3 database for Podcast Producer. This database is stored in the Server directory of your shared library. Within this database there is a table called episodes. Using that table you can locate all episodes that match the given pattern. To query, you will use the sqlite3 command and identify the database path. A very basic incantation of this command would be to show all of the data for a given table, which in our example would be “episodes”: sqlite3 /Volumes/xsanVolume/Library/PodcastProducer/Shared/Server/db.sqlite3 ‘SELECT * FROM episodes’ You could then expand this to limit results based on a pattern match. Here, we will have sqlite3 return the uuid for episodes that have a workflow_uuid of AAAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEEEE: sqlite3 /Volumes/xsanVolume/Library/PodcastProducer/Shared/Server/db.sqlite3 ‘SELECT uuid FROM episodes WHERE workflow_uuid=”AAAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEEEE”‘ We could also change it up slightly and look for episodes that were created by a user with a shortname of cedge: sqlite3 /Volumes/xsanVolume/Library/PodcastProducer/Shared/Server/db.sqlite3 ‘SELECT uuid FROM episodes WHERE author_shortname=”cedge”‘ The sqlite3 commands will return a set of uuids. Once you have a list of uuids, you can go into the Content directory in your shared library and you will see each listed there based on the date they were created (inside of their date folders). Date is also a field in the sqlite3 database, but given the ease of recursively performing tasks I’m not sure it will be required. The uuids will have a .prb extension and you can then piped paths with the added extension out of your command, into an array or use them for some other action, thus allowing for archiving, removing and even performing further tasks to the assets that live within the actual bundle, including hooking into transmogrifier to link to Final Cut Server.

Managing SyncPlans from the PresSTORE CLI

PresSTORE has a Synchronize module, that can be used to copy data from one location to another. This is done by the use of synchronization plans, or sync plans for short. Each plan is given a name and has a number of attributes associated with it, such as whether it is enabled or disabled. PresSTORE has a command line interface called nsdchat that is available at /usr/local/asw. To run it in interactive mode you can run the following command:
From here, you can use the SyncPlan command to interface with the plans that you have created. To see a list of plans you will use the SyncPlan command from the interactive mode, along with the names method:
SyncPlan names
Once you see a list of defined sync plans you can use the describe method to obtain an explanation of what each is comprised of. Let’s say we have a sync plan called XsanSync. If you wanted to see a human readable description of what it does, you could then use the following:
SyncPlan XsanSync describe
To enable one of your sync plans use the enable method. For example, to continue on with the plan called XsanSync you can use the following command to enable it:
SyncPlan XsanSync enable
To verify that the plan was enabled, use the enabled method. This past tense is only used to return a 1 for enabled or a 0 for disabled:
SyncPlan XsanSync enabled
Note: The disabled method can be used to achieve the opposite result. The sourcehost and targethost methods can be used to indicate what computer is having data synchronized from and which is having data synchronized to. The sourcepath and targetpath methods can then be used to indicate what path on the sourcehost and targethost is being sync’d, respectively. You cannot then change the host or path of either using the SyncPlan command. Once you’re comfortable with what is being sync’d and why, you can fire off a sync job, either immediately or send it to the PresSTORE scheduler. To submit a job to run immediately, use the submit method along with the now option like so:
SyncPlan XsanSync submit now
Running the same command without the now simply submits the job to the scheduler based on the schedule in the plan. To stop a running job, the aptly named stop method will be used:
SyncPlan XsanSync stop
The stop method can only be used on a running sync plan, and so if you have a job that has been submitted to the scheduler that you would like to cancel, you would use the cancel method:
SyncPlan XsanSync cancel
The -c (BTW, thanks Beau for showing me what I was doing wrong with -c) can be used to run nsdchat in a non-interactive mode. For example, to cancel the SyncPlan job referenced above, you can do the following:
/usr/local/aw/nsdchat -c SyncPlan XsanSync cancel
Overall, the ability to script logic (bash, AppleScript, etc) for sync plans based on system events, 3rd party apps (Podcast Producer and Final Cut Server for starters) or anything else you can obtain a variable for provides a lot of versatility and extensibility to the out-of-the-box PresSTORE solution for data synchronization. Granted, you can do any of the aforementioned commands by using either the web GUI or the iPhone app. But, as usual, we don’t need no stinkin’ GUI (even if the PresSTORE 4 GUI is a lot better than in previous versions)!

Stupid Podcast Producer Tricks

The document handler in Podcast Producer has been exposed to the command line in the form of a tool called document2images (located in the /usr/libexec/podcastproducer directory), which takes a pdf and converts it into a set of tiff files. In its most basic iteration the documents2images tool simply inputs a document and then outputs a couple of tiff files per page of that document. 15 pages will typically net you 30 tiffs and an xml output (not that you can put Humpty Dumpty back together again very easily). When you use document2images you will need to specify the pdf using the –document option, the xml file to output using the –xml option and the directory to drop your images into using the –imagespath option. To use an example of this command, if I wanted to convert a pdf called /Users/cedge/Desktop/test.pdf into images in the /Users/cedge/Desktop/tiffs directory and drop the XML file into /Users/cedge/Desktop I would use the following command:
/usr/libexec/podcastproducer/document2images –document /Users/cedge/Desktop/test.pdf –xml /Users/cedge/Desktop/test.xml –imagespath /Users/cedge/Desktop/test
It’s worth note that the user invoking the command will need access to write to the directory that you’re dropping your images into as well as the directory that the xml file will be written to (and of course, to read the pdf).

Podcast Producer on Your iPhone

PCP Remote is a great little application for initiating Podcast Producer workflows from your iPhone. In the current iteration you can also record audio tracks from the iPhone and use them in workflows as well. Oh, and it can connect to multiple Podcast Producer servers to kick off workflows. Since one workflow can summon another or write data into Final Cut Server, this gives a lot of options for different ways to integrate Podcast Producer and automate a number of items from your iPhone. Great stuff and I for one am really looking forward to the next version as soon as the App Store approves it!

Podcast Producer Error Codes

When you are using Podcast Producer, whether you are looking in your log files on a server in Server Admin or whether you are looking in Console on a client, if there are any problems submitting jobs you should find a numeric reference code. The meaning can be cryptic, although I’ll try a little bit here:
-500 = Camera agent went offline -501 = Podcast Producer agent timed out -502 = Agent failed to communicate -600 = Tunnel protocol mismatch -601 = Tunnel could not connect -602 = Tunnel timed out -603 = Tunnel failed 1 = Internal camera agent is failing 3 = Capture failed to run 4 = Capture already running 5 = Capture is not paused 6 = The locking mechanism encountered a mutex failure 7 = The unlocking mechanism encountered a mutex failure 8 = Camera agent cannot communicate with the file system 9 = Podcast Capture has no capture device available 10 = SG initialization failure 11 = Audio channel initialization failure 12 = Video channel initialization failure 13 = Video creation failed 14 = QuickTime encountered an Error 15 = QuickTime SG failed to start 16 = QuickTime SG failed to prepare 17 = QuickTime SG failed to set the data processor 18 = QuickTime compression session failed 19 = QuickTime Decompression session failed 20 = QuickTime SGIdle failed to initialize 21 = QuickTime data failed to process 22 = QuickTime sound failed to compress 23 = Writing the movie frame failed 24 = Could not start the process (a bit cryptic) 25 = Could not stop the process (a bit cryptic) 26 = Could not finalize the movie 27 = Action not supported 28 = Action could not resume processing 29 = OpenGL error 30 = Insufficient disk space for the action (or could not reach file system) 31 = Invalid/incorrect command 32 = Screen grab functionality disabled

Cleaning Up Podcast Producer

THIS CAN BE A VERY DANGEROUS COMMAND, USE IT AT YOUR OWN RISK. I’ve been seeing Podcast Producer deployed into a number of different types of environments. One of these is into school environments with quarterly turnover cycles. In these environments, all blog posts over a certain number of days old should be deleted routinely (namely when the quarter changes). Because in some environments, there can be hundreds, if not thousands, of posts, removing old content is cumbersome at best. Never fear, the find command can look at just items that have aged a certain number of days. This is accomplished using the -mtime options. The find command can also execute a command. By default a blog will be stored in a folder called /Library/Collaboration/Users//weblog directory. If you replace the aforementioned with a users name or the /Users with Groups and place a group blog name in the place of then you can view the files that make up the posts. Combined you can clear out old content from the wiki (be it 7 years or 7 days). In the following example, we’re going to delete all contents from the krypted user blog that is older than 90 days:
find /Library/Collaboration/Users/krypted/weblog -mtime +90 -exec rm. {} ;
The same command can then be used with the path of your video repository rather than the weblog. One will remove the videos, the other the impact of those posts to user experience.

pcastaction verbs

pcastaction comes with a number of verbs, each specific to a type of automation that can be used in Podcast Producer. These include:
  • unpack – extract a folder archive before running the automation
  • shell – run a command or shell script
  • preflight – run a script before the automation
  • postflight – run a script after an automation
  • encode – input a standard video file and then output a video file using a different codec
  • annotate – annotate a files metadata
  • qceffect – run a custom Quartz Composer composition against a file
  • watermark – insert a watermark into an indicated video file
  • title – provide a title for the resultant file
  • merge – merge two existing files
  • iTunes – indicates the video is to be included in an iTunes RSS feed
  • iTunesU – interface with iTunesU the same way that the iTunes verb can do
  • mail – send an email announcement about the new video
  • archive – archive files used in the automation
  • publish – publish the required files into the root of the web server
  • groupblog – add the item into the group’s RSS feed
  • template – create a new file from a template
  • approval – submit content for approval