Waveform Focus

This chapter details a method of keeping the waveform of the most recently started Audio cue or Video cue visible. This is useful for working with audio for productions that have not been fully rehearsed, allowing a designer or operator to see whether manual intervention may be required to adjust the volume on the spot, to get some indication about possible cue points when busking lights, and so on.

Here it is in action:

In this workspace, the playhead (indicated by the white pointer triangle in the far left column of the cue list) and the selection are unlocked, so the selection can be set independently of the playhead.

Cue 1 is selected by clicking in the status column on the left edge of the cue list.

When the workspace GOes, the following things happen:

• Cue 1 starts.
• The playhead moves to cue 2.
• The selection moves to cue 1 because it is the last Audio cue started.
• The workspace is switched into edit mode.
• The inspector opens and switches to the Time & Loops tab to display the waveform.
• The waveform is zoomed out fully to ensure the playhead cursor is not off the screen and that the entire waveform is displayed.

Cue 2 is a Timeline Group cue containing an Audio cue and a Video cue without a soundtrack. When it is played, the Audio cue becomes the selected cue because it contains the most recent audio played.

In Cue 4, the Video cue does have a soundtrack. Still, the Audio cue is what ends up selected because it is the most recent audio played.

Cue 5 is identical to Cue 4, except the order of the Video and Audio cues are reversed. In this case the Video cue is selected because its soundtrack is the most recent audio played.

After Cue 6 is run, the playhead is set to cue 1 again by clicking in the status column. This also moves the selection. The inspector is switched back to displaying the waveform of the most recently started Audio cue by pressing 1 on the keyboard.

How It Works

First, the playhead must be unlocked from the selection. This is achieved by going to Workspace Settings → General and un-checking the box labeled Lock Playhead To Selection.

If you usually operate with the playhead locked to the selection, you will need to get used to remembering that clicking on a cue or using the up and down arrow keys will only move the selection, not the playhead. You can use ⇧⌘↑ and ⇧⌘↓ or click in the status column to move the playhead. This does take some getting used to; a post-it note on the screen can be a useful reminder until it becomes second nature!

The next thing this workspace does is “hijack” the space bar so that its normal behavior can be changed. To do this, it is removed as the keyboard shortcut for GO in Workspace Settings → Controls → Keyboard.

A cue list named “Scripts” contains a Script cue numbered “GO”, which has its hot key trigger set to the space bar. This is done in the Triggers tab of the inspector:

You can also use a MIDI trigger here as an alternative.

The script of this cue contains all the programming necessary to perform the special actions that occur in this workspace. Here it is:

--GO and select last  audio cue played in second inspector window
tell application id "com.figure53.QLab.5" to tell front workspace
  go
  set show mode to false
  set inspector visibility to true
  delay 0.1
  try
    set thecue to last item of (cues whose running is true and (q type is "audio" or q type is "video") and audio input channels is greater than 0)
    set the selected to thecue
    set theCueList to q name of parent list of thecue
  on error
    display dialog "no valid cue selected"
    return
  end try
  --UI Scripting to select time & Loop tab of inspector and zoom out waveform fully
  tell application "System Events" to tell application process "QLab" to tell (first window whose description contains "workspace window") to tell splitter group 1 to tell (first group whose description contains "cue inspector")
    click (first button whose description contains "Time & Loops")
    repeat until (exists scroll bar of scroll area 1) is false
      click (first button whose help contains "Zoom Out")
    end repeat
  end tell
end tell
  if playhead of current cue list is (last item of (selected as list)) then display alert "Playhead is locked to selection" giving up after 5
end tell

The script begins by telling the workspace to GO, which causes the current cue list to start the cue at the playhead and then move the playhead down as appropriate. This is done first so that the workspace remains (very very nearly) as responsive as a regular workspace.

It then uses terms from QLab’s AppleScript dictionary to enter edit mode and display the inspector.

It finds the last Audio or Video cue played with more than 0 audio input channels. This means that a Video cue without an audio track will not be selected.

We now need to select the Time & Loops tab of the inspector for the selected cue and zoom the waveform out to show its entire duration. At this point, we have reached a major AppleScript hurdle; there are no commands in QLab’s AppleScript dictionary to achieve either of these tasks.

All is not lost because we now resort to UI Scripting, an often overlooked pocket of the AppleScript universe, which allows us to use AppleScript to directly control the GUI (Graphical User Interface) elements as if we were performing the actions by hand with a mouse. The script simulates the mouse clicking on the Time & Loops tab and then repeatedly clicking the button with the zoom-out icon until there is no scroll bar under the waveform.

To use UI Scripting in QLab, you first have to grant permission for QLab to receive UI scripting actions. You do this by visiting System Preferences → Security & Privacy → Accessibility (in macOS versions before Ventura) or System Settings → Privacy & Security → Accessibility (in macOS versions Ventura or later) and adding QLab to the list.

UI Scripting is a bit of dark art, made even more difficult because the best tool for finding UI elements, UI Browser by Bill Cheeseman, is no longer maintained and licenses are no longer for sale. Even so, this chapter uses screenshots from UI Browser to demonstrate some principles of UI scripting in QLab. Identifying the right AppleScript name for UI elements without the help of UI Browser typically involves using Accessibility Inspector, part of the Xcode developer tools from Apple, or other accessibility tools that provide information about the elements of an application’s user interface. Examples using these tools are given later in this tutorial. It’s not practical to provide a full description of using these tools for UI scripting in a short chapter like this, but numerous learning resources on the web are available.

You can download UI Browser here and use it in trial mode for 30 days.

Here’s the UI scripting path for the Time & Loops tab shown in UI Browser when an Audio cue is selected in QLab:

We can see the terms necessary to identify this button to AppleScript in the bracketed names in the Element Path. So, we can script the action of clicking on this tab like this:

tell application "System Events" to tell application process "QLab" to click button 5 of group 1 of splitter group 1 of window 1. 

It doesn’t really roll off the tongue, but it works.

However, here is the Time & Loops tab shown in UI Browser when a Video cue is selected in QLab:

The tab is now Button 6, not Button 5. We could get around this discrepancy by having our script determine the q type and using Button 5 or Button 6 as appropriate, but there is a further problem: Even though the workspace window has keyboard focus, it has now become Window 2 because the playing video is now Window 1. If we had another window open as well, such as the Workspace Status window, then the Workspace window would be window 3. So, we need to find a better way of identifying UI Elements.

If we look at the workspace window in UI Browser, we can see above the Element Path that there are other properties that can be used to identify a window.

The Role description, Role, and Subrole aren’t useful because many QLab windows have similar roles. The Title is specific to the workspace and the selected cue list, so it isn’t universally applicable in a script. The Description, however, includes the string “workspace window”, which is exactly what we need to identify this window regardless of any other properties.

If you look again at the Time & Loop screen shots above, you will see that their Description properties contain the string “Time & Loops” in both images. Note, however, that in the Video cue screenshot where the tab is already open, the Description also adds the word “selected”, so we need to ensure we test whether the Description contains the string “Time & Loops” rather than whether it is “Time & Loops”.

In this screenshot, the cursor is hovering over an unused part of the inspector, which reveals the Element Path and property names of the Group Element. In this context, “group” means a group of controls on the screen, like buttons, tabs, text areas, and so on.

The Description of this UI element contains the string “cue inspector”.

So we can see that the Description property of an element often contains a string that will unambiguously identify the element we wish to script.

tell application "System Events" to tell application process "QLab" to click (first button whose description contains "Time & Loops")of (first group whose description contains "cue inspector") of  splitter group 1 of (first window whose description contains "workspace window")

This can also be scripted in reverse order of elements as:

tell application "System Events" to tell application process "QLab" to tell (first window whose description contains "workspace window") to tell splitter group 1 to tell (first group whose description contains "cue inspector") to tell (first button whose description contains "Time & Loops") to click

We’ll now look at how the Time & Loop tab will appear if you use Apple’s Accessibility Inspector to find the UI Element Descriptions.

The selected UI element is shown with a diamond mark in the hierarchy.

Working up from this you can see that the first part of each line is the same description that was shown in UI Browser. So, as long as you are using the descriptions to identify the elements, the Accessibility Inspector is a very usable method of obtaining them.

As with the UI Browser screenshots, however, you will notice that there is no Description for the splitter group because it doesn’t have one. Generally, splitter groups are numbered 1, but while the UI browser showed the element as “Splitter Group”, which is the term AppleScript understands, all of Apple’s developer tools identify this element as a “split group”, which AppleScript will fail to compile. As long you use “splitter group 1” in your script anytime a split group appears in a UI element hierarchy, then it should work.

Sometimes, a button with a missing “Description” may be addressable via its “Help” property. The Help property is the text that will be displayed if the cursor is hovered over the button.

This is the case with the zoom out button that we scripted to click repeatedly zoom out on the waveform until the entire waveform was displayed:

repeat until (exists scroll bar of scroll area 1) is false
  click (first button whose help contains "Zoom Out")
end repeat

The help property of a UI element can be found in both UI Browser and Accessibility Inspector.

Another way to figure out how to address a given UI element is to run this script in Script Editor:

tell application "System Events" to tell application process "QLab"
  set theList to buttons of (first group whose description contains "cue inspector") of splitter group 1 of (first window whose description contains "workspace window")
  set theCount to count of items of theList
  set theDescriptions to {}
  repeat with i from 1 to theCount
    set the end of theDescriptions to item i in theList
    set the end of theDescriptions to "DESCRIPTION: " & description of item i of theList
    set the end of theDescriptions to "HELP: " & help of item i of theList
  end repeat
  return theDescriptions
end tell

The results aren’t particularly pretty, but the number, description and help text for every button in the inspector is listed.

You can modify this script to find other UI elements by changing “button” to “scroll area”, “checkbox”, “radio button”, etc. Before you put too much time into this, though, remember that most settings in the inspector have commands in QLab’s AppleScript dictionary, and it is much better to use those wherever possible.

The final part of the script discovers whether the playhead is locked to the selection by testing if the selection and the playhead are the same after running the script. If they are, the script displays an alert. This can be dismissed manually, or it will disappear automatically after five seconds.

  if playhead of current cue list is (last item of (selected as list)) then display alert "Playhead is locked to selection" giving up after 5

A second Script cue triggered with hotkey 1 is identical to the script triggered by the space bar, except it doesn’t include the command to GO; it just selects the last Audio cue (or Video cue) played, opens the inspector, navigates to the Time & Loops tab, and zooms out the waveform. It’s most useful if the selection changed because you moved the playhead to a different cue and you want to return to the currently playing waveform.

--Select last  audio cue played in second inspector window
tell application id "com.figure53.QLab.5" to tell front workspace
  set show mode to false
  set inspector visibility to true
  delay 0.1
  try
    set thecue to last item of (cues whose running is true and (q type is "audio" or q type is "video") and audio input channels is greater than 0)
    set the selected to thecue
    set theCueList to q name of parent list of thecue
  on error
    display dialog "no valid cue selected"
    return
  end try
  --UI Scripting to select time & Loop tab of inspector and zoom out waveform fully
  tell application "System Events" to tell application process "QLab" to tell (first window whose description contains "workspace window") to tell splitter group 1 to tell (first group whose description contains "cue inspector")
    click (first button whose description contains "Time & Loops")
    repeat until (exists scroll bar of scroll area 1) is false
      click (first button whose help contains "Zoom Out")
    end repeat
  end tell
end tell
  if playhead of current cue list is (last item of (selected as list)) then display alert "Playhead is locked to selection" giving up after 5
end tell

All music in the demo workspace by Kevin MacLeod. Licensed under Creative Commons: By Attribution 3.0