• Solutions
    • FERC XBRL Reporting
    • FDTA Financial Reporting
    • SEC Compliance
    • Windows Clipboard Management
    • Legato Scripting
  • Products
    • GoFiler Suite
    • XBRLworks
    • SEC Exhibit Explorer
    • SEC Extractor
    • Clipboard Scout
    • Legato
  • Education
    • Training
    • SEC and EDGAR Compliance
    • Legato Developers
  • Blog
  • Support
  • Skip to blog entries
  • Skip to archive page
  • Skip to right sidebar

Friday, June 16. 2017

LDC #39: Application Initialize Script

So far, every script we’ve explored has been a hook of some sort. They attach themselves to new or existing menu icons or are triggered by other functions being run. What if you want something to run on application start? Legato supports this through the Application Initialize script. On program start, GoFiler will always run the ApplicationStartup.ls script. This script file is the one that’s responsible for running every other script in the extensions folder. This allows users to create their own custom startup routines that are run before literally anything else.

Today, our example will be the ApplicationInitialize.ls file, configured to check to make sure the user is running the most up to date CUSIP list for 13-F validation. This is going to be included in future versions of GoFiler Complete and Go13, but it makes a good example blog post as well.


The script:


//
//      GoFiler Complete - Initialize Application
//      ------------------------------------
//
//      Runs on opening application. Currently checks 13F CUSIP list.
//
//      Revised         06/13/2017      Initial
//
//
//      (c) 2017 Novaworks, LLC. All rights reserved.

#define                 CUSIP_LIST_BASE                 "https://www.sec.gov/divisions/investment/13f/13flistYYYYqQQQQ.pdf"
#define                 TEMP_FILENAME                   "temp_cusips.pdf"
#define                 CUSIP_CSV                       "cusips.csv"
#define                 LAST_ATTEMPT                    "Last CUSIP Download Attempt"
#define                 ONE_DAY                         864000000000

    int                 check_cusip_list                ();             /* check the current CUSIP list         */
                                                                        /****************************************/
int main(){                                                             /* runs on startup                      */
                                                                        /****************************************/
    check_cusip_list();                                                 /* run the check cusip function         */
    return ERROR_NONE;                                                  /* return without error                 */
    }                                                                   /*                                      */
                                                                        /****************************************/
int check_cusip_list(){                                                 /* check that the cusip list is current */
                                                                        /****************************************/
    qword               time;                                           /* current time                         */
    qword               last_attempt;                                   /* last attempted download              */
    int                 rc;                                             /* result                               */
    string              time_s;                                         /* time as a string                     */
    string              product;                                        /* name of product                      */
    string              appdata;                                        /* location of appdata folder           */
    string              settings;                                       /* gofiler's settings file              */
    string              cusip_file;                                     /* cusip file                           */
    string              year;                                           /* year                                 */
    string              month;                                          /* month of the year                    */
    string              quarter;                                        /* quarter of year as a string          */
    string              old_cusip_csv;                                  /* the old cusips CSV file              */
    string              cusip_loc;                                      /* location of cusip file to try        */
    string              library_loc;                                    /* library location on machine          */
    string              tempfile;                                       /* temporary cusip file                 */
    string              current_list;                                   /* current cusip list                   */
                                                                        /*                                      */
    time = GetLocalTime();                                              /* get the local time                   */
    month = MakeLowerCase(FormatDate(time,"M"));                        /* get current month                    */
    if (month == "jan" || month == "feb" || month == "mar"){            /* if q1                                */
      quarter = "4";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    if (month == "apr" || month == "may" || month == "jun"){            /* if q2                                */
      quarter = "1";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    if (month == "jul" || month == "aug" || month == "sep"){            /* if q3                                */
      quarter = "2";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    if (month == "oct" || month == "nov" || month == "dec"){            /* if q4                                */
      quarter = "3";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    year = FormatDate(time,"o");                                        /* get year                             */
    if (quarter == "4"){                                                /* if we want Q4's list                 */
      year = FormatString("%d",TextToInteger(year)-1);                  /* subtract a year                      */
      }                                                                 /*                                      */    
    cusip_loc = ReplaceInString(CUSIP_LIST_BASE,"YYYY",year);           /* add year to cusip string             */
    cusip_loc = ReplaceInString(cusip_loc,"QQQQ",quarter);              /* add quarter to cusip string          */
    appdata = GetApplicationDataFolder();                               /* get the appdata folder               */
    product = GetApplicationName();                                     /* get the application name             */
    settings = product + " Settings.ini";                               /* get name of settings file            */
    settings = AddPaths(appdata,settings);                              /* get path to settings                 */
    current_list = GetSetting(settings,"Form 13F Options","PDF CUSIPs");/* get current cusip list               */
    library_loc = GetSetting(settings,"EDS","Library");                 /* get library location                 */
    old_cusip_csv = AddPaths(library_loc,CUSIP_CSV);                    /* get path to old cusip csv file       */
    if (current_list == cusip_loc){                                     /* if we have the most up to date list  */
      return ERROR_NONE;                                                /* return without error                 */
      }                                                                 /*                                      */
    last_attempt = GetSetting(settings,"Form 13F Options",LAST_ATTEMPT);/* get last attempt to dl file          */
    if ((time - last_attempt)<ONE_DAY){                                 /* if less than one day has passed      */
      return ERROR_NONE;                                                /* return without trying again          */
      }                                                                 /*                                      */
    tempfile = AddPaths(GetTempFileFolder(),TEMP_FILENAME);             /* get path to temp file                */
    rc = HTTPGetFile(cusip_loc,tempfile);                               /* get temp file                        */
    DeleteFile(tempfile);                                               /* remove temp file                     */
    if (rc!=200){                                                       /* if we failed to get the file         */
      PutSetting(settings,"Form 13F Options",LAST_ATTEMPT,time_s);      /* store last attempted dl time         */
      return ERROR_NONE;                                                /* return without error                 */
      }                                                                 /*                                      */
    PutSetting(settings,"Form 13F Options","PDF CUSIPs",cusip_loc);     /* store new cusip list                 */
    DeleteFile(old_cusip_csv);                                          /* delete the old CUSIP CSV file        */
    LoadPreferences();                                                  /* force preferences to reload          */
    return ERROR_NONE;                                                  /*                                      */
    }                                                                   /*                                      */


The first thing to notice about this script is that it’s much simpler in structure than other ones in the past. It simply has two functions: main and check_cusip_list. It really doesn’t need anything other than main, but if other initialization functions need to be added later to the ApplicationInitialize.ls file, it helps to have the script organized into different functions for each action that needs to be performed. There is no need for a setup function because the main function is automatically run every time the application starts up.


#define                 CUSIP_LIST_BASE                 "https://www.sec.gov/divisions/investment/13f/13flistYYYYqQQQQ.pdf"
#define                 TEMP_FILENAME                   "temp_cusips.pdf"
#define                 CUSIP_CSV                       "cusips.csv"
#define                 LAST_ATTEMPT                    "Last CUSIP Download Attempt"
#define                 ONE_DAY                         864000000000


The script uses five different defines. CUSIP_LIST_BASE is the template location for CUSIP lists on the SEC’s website. The YYYY value in the string needs to be replaced with the correct year, and the QQQQ value needs to be replaced with the correct quarter. The TEMP_FILENAME is the name of the file we will try to download off the SEC’s site. It’s temporary because we only need to download it to check if it is available, after which it can be deleted. The CUSIP_CSV define is GoFiler’s internal CUSIP library file. After we update the list, we need to delete this to force GoFiler to rebuild it with the new list. LAST_ATTEMPT stores the last time a CUSIP download was attempted and failed to ensure we don’t spam the SEC’s website every time GoFiler opens to request a new CUSIP list that doesn’t exist. ONE_DAY is one day in Windows filetime format.


    time = GetLocalTime();                                              /* get the local time                   */
    month = MakeLowerCase(FormatDate(time,"M"));                        /* get current month                    */
    if (month == "jan" || month == "feb" || month == "mar"){            /* if q1                                */
      quarter = "4";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    if (month == "apr" || month == "may" || month == "jun"){            /* if q2                                */
      quarter = "1";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    if (month == "jul" || month == "aug" || month == "sep"){            /* if q3                                */
      quarter = "2";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    if (month == "oct" || month == "nov" || month == "dec"){            /* if q4                                */
      quarter = "3";                                                    /* use previous quarter                 */
      }                                                                 /*                                      */
    year = FormatDate(time,"o");                                        /* get year                             */
    if (quarter == "4"){                                                /* if we want Q4's list                 */
      year = FormatString("%d",TextToInteger(year)-1);                  /* subtract a year                      */
      }                                                                 /*                                      */    
    cusip_loc = ReplaceInString(CUSIP_LIST_BASE,"YYYY",year);           /* add year to cusip string             */
    cusip_loc = ReplaceInString(cusip_loc,"QQQQ",quarter);              /* add quarter to cusip string          */


The first thing our script needs to do is build the location to check for the file. We can use the GetLocalTime SDK function to get the current time. Using the FormatDate function, we can get the current month as a 3 letter abbreviation, and make sure it’s lower case with the MakeLowerCase function. If it’s the first 3 months of the year, we want to use Q4’s list. Otherwise, we just set quarter to the previous quarter in this year. We can use the FormatDate function to get the year as well. After we have the current year, we need to check if we’re getting Q4’s list. If so, we want to subtract a year from our current year. Because year is a string, we need to convert it to an integer value first with the TextToInteger function and then back to a string with the FormatString function. Once we have both the quarter and year, we can use the ReplaceInString function to build the location to test on the SEC’s site.


    appdata = GetApplicationDataFolder();                               /* get the appdata folder               */    
    product = GetApplicationName();                                     /* get the application name             */
    settings = product + " Settings.ini";                               /* get name of settings file            */
    settings = AddPaths(appdata,settings);                              /* get path to settings                 */
    current_list = GetSetting(settings,"Form 13F Options","PDF CUSIPs");/* get current cusip list               */
    library_loc = GetSetting(settings,"EDS","Library");                 /* get library location                 */
    old_cusip_csv = AddPaths(library_loc,CUSIP_CSV);                    /* get path to old cusip csv file       */
    if (current_list == cusip_loc){                                     /* if we have the most up to date list  */
      return ERROR_NONE;                                                /* return without error                 */
      }                                                                 /*                                      */


Next, we need to get our settings file name so we can check if we need to update the CUSIP list or not. Using the GetApplicationDataFolder and GetApplicationName functions, we can build a path to the settings file so we can read settings out of it. We’re also going to need to get the path to the old CUSIP file, which is in the library directory. So we pull the library directory out of the settings file and add it to our defined file name to get the path to the library file. Then, using the GetSetting function, we need to check the setting named “PDF CUSIPs” to see if it matches our generated location on the SEC’s site. If it matches, we already have the most up to date list, so we can just return without error.


    last_attempt = GetSetting(settings,"Form 13F Options",LAST_ATTEMPT);/* get last attempt to dl file          */
    if ((time - last_attempt)<ONE_DAY){                                 /* if less than one day has passed      */
      return ERROR_NONE;                                                /* return without trying again          */
      }                                                                 /*                                      */
    tempfile = AddPaths(GetTempFileFolder(),TEMP_FILENAME);             /* get path to temp file                */
    rc = HTTPGetFile(cusip_loc,tempfile);                               /* get temp file                        */
    DeleteFile(tempfile);                                               /* remove temp file                     */
    if (rc!=200){                                                       /* if we failed to get the file         */
      PutSetting(settings,"Form 13F Options",LAST_ATTEMPT,time_s);      /* store last attempted dl time         */
      return ERROR_NONE;                                                /* return without error                 */
      }                                                                 /*                                      */
    PutSetting(settings,"Form 13F Options","PDF CUSIPs",cusip_loc);     /* store new cusip list                 */
    DeleteFile(old_cusip_csv);                                          /* delete the old CUSIP CSV file        */
    LoadPreferences();                                                  /* force preferences to reload          */
    return ERROR_NONE;                                                  /* return without error                 */


If our file is out of date, we can get the last attempted download with the GetSetting function. If it’s less than one day ago, we want to return without error again, because we already checked today. Otherwise, we can build a path to a temp download file. Then we can use the HTTPGetFile function to try to download the file using our link to the temp file directory. Immediately after download, we can just delete the file, because the actual contents of the file are irrelevant. All we care about is whether or not we could download the file. For that, we just need to check the return code of the HTTPGetFile function. Return code 200 means that everything is OK, and we got the file successfully. Anything else means an error happened and we cannot download the file successfully, so we need to store the last attempt in the settings file before returning.


If we did download the file without a problem, we can again use the PutSetting function to put it into our settings file and then delete the old CUSIP file to force the application to re-download and parse it when the user next presses validate on a CUSIP list. Finally, we can call the LoadPreferences function to force our settings to re-load. This initialization function has GoFiler check every time it’s run to see if it has the most recent CUSIP file from the SEC, and if not, it attempts to download it. This a very appropriate use of the Application Initialization script to keep our application up to date.


 


Steven Horowitz has been working for Novaworks for over five years as a technical expert with a focus on EDGAR HTML and XBRL. Since the creation of the Legato language in 2015, Steven has been developing scripts to improve the GoFiler user experience. He is currently working toward a Bachelor of Sciences in Software Engineering at RIT and MCC.

Additional Resources

Novaworks’ Legato Resources

Legato Script Developers LinkedIn Group

Primer: An Introduction to Legato 



Posted by
Steven Horowitz
in Development at 17:19
Trackbacks
Trackback specific URI for this entry

No Trackbacks

Comments
Display comments as (Linear | Threaded)
No comments
Add Comment
Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
Standard emoticons like :-) and ;-) are converted to images.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

 
   
 

Quicksearch

Categories

  • XML Accounting
  • XML AICPA News
  • XML FASB News
  • XML GASB News
  • XML IASB News
  • XML Development
  • XML Events
  • XML FERC
  • XML eForms News
  • XML FERC Filing Help
  • XML Filing Technology
  • XML Information Technology
  • XML Investor Education
  • XML MSRB
  • XML EMMA News
  • XML FDTA
  • XML MSRB Filing Help
  • XML Novaworks News
  • XML GoFiler Online Updates
  • XML GoFiler Updates
  • XML XBRLworks Updates
  • XML SEC
  • XML Corporation Finance
  • XML DERA
  • XML EDGAR News
  • XML Investment Management
  • XML SEC Filing Help
  • XML XBRL
  • XML Data Quality Committee
  • XML GRIP Taxonomy
  • XML IFRS Taxonomy
  • XML US GAAP Taxonomy

Calendar

Back May '25 Forward
Mo Tu We Th Fr Sa Su
Tuesday, May 13. 2025
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31  

Feeds

  • XML
Sign Up Now
Get SEC news articles and blog posts delivered monthly to your inbox!
Based on the s9y Bulletproof template framework

Compliance

  • FERC
  • EDGAR
  • EMMA

Software

  • GoFiler Suite
  • SEC Exhibit Explorer
  • SEC Extractor
  • XBRLworks
  • Legato Scripting

Company

  • About Novaworks
  • News
  • Site Map
  • Support

Follow Us:

  • LinkedIn
  • YouTube
  • RSS
  • Newsletter
  • © 2024 Novaworks, LLC
  • Privacy
  • Terms of Use
  • Trademarks and Patents
  • Contact Us