Can you make a script that just “converts” a document with the styles I want with a single button press?

This is a fairly common question I’m asked in regards to Legato. I’ve had several clients request such a script, something where they would run an extension that would handle conversion of the document, and perform a number of additional tasks such as: polishing the tables to their specification, adding exhibit hyperlinking, applying font styles, and automatically adjusting page breaks. All of these things are certainly possible with Legato. I could write a script that converts a document, and scans through the tables, before determining if it’s a financial table or not and appropriately applying the table polish function. I could have a script that automatically scans an exhibit index and tries to link the exhibits to other files in the same filing. Even though each piece can be done, a script that does ALL of this with a single button press is not really practical. Let’s consider why that might be.

Such an all encompassing script would have a large branching decision tree that the script itself would have to keep track of. For example, perhaps your document “style” you’re trying to apply requires financial tables to be polished with table gutters and striping, and requires financial pages to have a page number like “F-1”, “F-2”, etc. Well, first your script would have to analyze a document to decide what pages are part of the financial tables section. It would need to make those decisions and store it for later, so when it goes to link an exhibit index it doesn’t get confused and try to add hyperlinks to a financial table. Whenever you try to have a script make decisions based on previous decisions it has made, it’s no longer a simple “if this, do this” decision tree. Now it has to keep track of past results, and make decisions based on those results. A script that performs two functions in this way is significantly more complex than two separate scripts that perform one single function each... because now you have to manage the interactions between these functions through a script. A script like the function described in the question above, that would just “convert” a document, would have to keep track of lots of decisions made, with a huge branching decision tree, and would be extremely complex.

Additionally, if our script’s target function is to convert Word documents to HTML documents, we need to consider HTML coding, and how variable it can be. Consider the following code snippets:

<DIV style="Display:Block"><FONT STYLE="font-family: Arial, Helvetica, Sans-Serif"><B><U>This is a sample of my code.</U></B></FONT></DIV>

<P STYLE="font-family: Arial, Helvetica, Sans-Serif; font-weight: bold; text-decoration: underline">This is a sample of my code.</P>

Both of these blocks of code will display very nearly the same exact way in an HTML browser like Chrome. However, as you can see they are coded drastically differently. One is a paragraph tag, one is a division tag. One has lots of inline styles, one handles everything through block tags. This adds even more complexity to a relatively simple task, since we need to include functions for deciding exactly what we are looking at, which makes a complex script like the one button conversion discussed above even more complicated.

Well, what kind of tasks can be automated with Legato? 

I generally like to describe functionality that is a good target for Legato automation as “mindless”. If you’ve got a task that requires relatively little decision making, then it’s a very good candidate for Legato automation – or for any other kind of automation. One of the very first examples we covered in this blog was replacing wingdings checkboxes with unicode checkboxes. This can be a time consuming task for a human, but for a machine it’s literally just scanning a document, finding a snippet of code, and replacing it with another. This is something I’d consider a mindless task, because we don’t have to have any decision making at all... we can just tell it to replace all X with Y. Easy.

Some of our more complicated scripts begin to examine decision making though. In our 95th blog post, we took a look at a script that gathers our footnote references, and moves our endnotes to the appropriate position in the document. The wingdings replace script is 150 lines of code, about half of that being actual logic to do the function, while the endnotes to footnotes conversion script is 500 lines of code, most of that being logic to make decisions. They both sound like relatively simple tasks... moving endnotes to the correct footnote location, and replacing windings font characters with unicode characters, but because our endnotes to footnotes script needs to analyze HTML and decide what is a footnote and what is an endnote, and remember that information, it is significantly more complex. The endnotes to footnotes converter is probably the most complicated type of script I’d suggest using Legato for... it analyzes HTML, and then makes simple decisions based on it. Even accounting for the overall “simplicity” of the script, it still look me the better part of a week to write that script and test it to be sure it worked for most HTML files.

Don’t forget the UI!

If we’ve identified the task as suitable for Legato automation, when we’re estimating the complexity, another major factor we need to consider is the UI, and exactly how much user interaction is going to be required. Legato has a few very basic dialogs built into it, like the YesNoBox, or the YesNoCancelBox functions, to get simple responses back from the user. However, if we’re going to require the user to interact on a more complex basis, we need to consider exactly what kind of interface we may need. If we need a custom control, we’ll need a resource file to describe what that control looks like, and every possible action a user can perform on that dialog will need to be controlled. One of our previous posts, the four part saga about bulk filings, ended up being 440 lines of code. Nearly half of that is just defining the behavior of the dialog controls that are available. It's not unreasonable to say that a large, complicated dialog control can literally double the development time of a script.

Avoiding biting off more than you can chew

Let’s put all this together. When considering creating a script for any given operation in Legato, things we should consider before we actually start mapping out the script should include:

1) Is this a “mindless” script, or will it require decision making? If it requires no decision making, it’s a great candidate for a script. If it requires decision making, it might not be a good candidate for writing a script. People are inherently good at making choices, while computer programs generally are not.

2) If it requires decision making, how much? Is it a simple choice, or will we need to remember several states? Exactly how much will our script need to keep track of? If we’re going to be basing decisions off of previous decisions our script has made, it’s going to increase the complexity of our script, and we run the very real risk of “failing” in our project. I’d consider a failed project one that takes significantly longer to write than it should, or only ends up working well part of the time. We want to avoid this.

3) Is our script trying to do too much? If our script is making decisions based on other decisions to run additional functions, maybe we’re better off breaking it into two or more scripts, that the user can choose to execute independently. Instead of polishing all tables in a document, and deciding which are financial and which are not and polishing them accordingly, maybe let the user select a section of the document and run one of two scripts on the selection of text. Breaking a large script apart into smaller ones will allow them to be deployed more quickly, and generally will make them more easily maintained and updated.

4) Consider how much the user is going to need to interact with this, and how we can keep it as minimal as possible. User interaction adds a lot of overhead to the script writing process. If we’re doing something that will inherently require a lot of interaction from our user, like modifying lists of things to be edited in a dialog box, it’s going to be much more complicated than simply having the user select a section of an HTML file and run a function on it. Be sure to consider what the requirements of the UI might be before committing to a project.

Keep all of the above in mind when deciding if a Legato script solution is appropriate for a given task you’re looking to automate. If you have a firm idea on exactly how complicated your script is going to potentially be before you write it, it can help ensure a successful project, and can help you estimate exactly how long the project is going to take. Otherwise, you could easily get mired down in a project that never seems to end, which is something nobody wants.

Steven Horowitz has been working for Novaworks for over five years as a technical expert with a focus on EDGAR HTML and XBRL. Since the creation of the Legato language in 2015, Steven has been developing scripts to improve the GoFiler user experience. He is currently working toward a Bachelor of Sciences in Software Engineering at RIT and MCC.

Additional Resources

Novaworks’ Legato Resources

Legato Script Developers LinkedIn Group

Primer: An Introduction to Legato