More on Scheduled Extract Automation

Illuminate Education can generate an automated extract using a Custom Report using our scheduled extract tool. You can access this tool in the Admin Cog > Data Extracts. As a reminder, we currently only support the movement of files using secure-file-transfer-protocol (SFTP).

Permissions

In order to use this feature, enable the following permission:

Data Extracts Access: Create, view, and edit data extracts

Features

On the Data Extracts page, you can accomplish the following tasks:

• View your scheduled extracts, including logs and server information on the Data Extracts list view.
• Tie automated scheduled extract jobs to Custom Reports.
• Manually run jobs.
• Update or delete jobs.

Data Extracts List View

The list view opens up from the Data Extracts menu option. It includes a table listing your jobs and a dropdown to create new jobs.

The table provides the following information:

• Type: This displays the name of the Custom Report and the session the scheduled extract job was created in.
• Name: This displays the name of the scheduled extract job set by the user.
• Config: This is reserved for certain system extracts. This is not used by the district user.
• Start On/Run After/Last Run Time: This generates a simple log of the day and time this extract was set-up to start and the last time the job ran.
• Site ID: This is reserved for certain system extracts. This is not used by the district user.
• Sever/Path/File: The server hostname, the directory path, and the filename set by the user.
• Power Icon: Displays a check mark if enabled. If it is disabled, it is blank the the job does not run.
• Edit: Edit link takes user to the Scheduled Extract form.
• Run: Runs the job manually. Below each Run button is an icon that will display a log in a modal.
• Delete: Deletes the scheduled extract job.

At the bottom of the page after the table, there will be a dropdown titled "Extract" and a create button. The dropdown produces a list of Custom Reports available to create an extract. Select a report and hit the create button to reach the job form. Hitting edit will take you to the form for the existing extract.

Below is a sample screenshot of this page:

Scheduled Extract Jobs

The following fields are available for the job form. Note the fields marked with a red asterisk are required and cannot be left blank.

• Extract Name: Pulls the name of the extract and the session the extract was made.
• Name: Job name.
• Server: Server's host name.
• Port: Typically 22, but can be changed based on the server.
• User Name: Account username to access the SFTP server.
• Password: Account password to access the SFTP server.
• Destination Path: Directory in the SFTP server.
• Use Absolute Path: By default, the destination path uses a relative path. You may need to use an absolute path based on the account permissions and directory structure.
• Filename: The file's name when generated.
• Delimiter: The file's delimeter. We currently support comma and tab.
• Line Break: Line Break when writing file. We recommend using a Carriage Return/Linefeed if you aren't sure.
• Field Surround Char: Surrounds each field's data with a string of character(s).
• Add Runtime: Allow you to add a start date and a time of day to run the extract using the Run After and Start On fields.
• Run After: Date to start running the jobs.
• Start On: Time to run the jobs daily for the runtime record.

• Exclude a Date Range: Allows you to set an exclusion date range to stop the job from running using the Exclude Start Date and Exclude End Date fields. Click this link causes "Exclude Start Date" and "Exclude End Date" fields to appear, along with the link to "Remove" the Exclusion Date Range See sample below:

• Exclude Start Date: Date to stop running the jobs.

• Exclude End Date: The last day to stop running jobs.
• Enabled: Enables or disables the job.
• Save: Submits the form.
• At the bottom of the page is a table displaying the columns from the Custom Extract tied to your job.

Note you can schedule several jobs with different filenames. For example, if you have an absence Custom Report, you can have one job generate in the morning at 10:00 AM called absences_10am.csv and another job create the file named absences_7PM.csv.

You can also elect to generate an absence file called absences.csv multiple times a day, at 10:00 AM which is then overwritten by the file later generated at 7:00 PM.