Friday, November 11, 2011

Speeding up the development lifecycle by selectively avoiding the database hit

Introduction
When coding in QlikView scripting engine, it takes time to run your code after each edit to determine if the script works or not.

If your script reads from a database e.g. DB2, the script execution can take some time as it waits on database results.

One way of speeding up the development lifecycle is use of a QlikView variable to inform the database not to expend any resources on actually running the SQL but instead return control back to QlikView to continue script execution.

The method shown below is just one of many options to reduce the time of execution. See later in this posting for other methods along with pros and cons.

Using SQL always-false condition

Imagine you have a LOAD SQL SELECT statement similar to this:
    table_name:
    LOAD
        ORDERID ;
    SQL

    SELECT
        ORDERID
    FROM
        STSWarehouse.dbo.CUSTORDER
    ;

Define a variable in the QVW script:
    LET v_sql_debug_condition='1=0' ;

Add this variable to your existing LOAD SQL SELECT:
    table_name:
    LOAD
        ORDERID 


;
    SQL
    SELECT
    ORDERID
    FROM
        STSWarehouse.dbo.CUSTORDER

    WHERE
        $(v_sql_debug_condition)
     ;

When the SQL is submitted to the database it will look like this:
    SELECT
    ORDERID
    FROM
    STSWarehouse.dbo.CUSTORDER
    WHERE
    1=0 


Since 1 is never equal to 0 (zero) the database will immediately recognize this and not expend any resources in executing the SQL or return any rows. It will also do so without generating any error messages and interrupting the flow of the script.

How do you allow the database to return rows?

Change the variable to a condition that is always true:
    LET v_sql_debug_condition='1=1' ;

And now when executing the same script, the SQL sent to the database will resemble:
    SELECT
    ORDERID
    FROM
    STSWarehouse.dbo.CUSTORDER

    WHERE
    1=1 


This always-true condition is recognized by the database and not considered in any execution path or explain plan i.e. will not interfere in the normal execution, selection or returning of rows.


Aren't there other ways to do this?
A similar reduction in script execution time can be achieved several other ways:
  1. Commenting out SQL LOAD statements to avoid them
  2. SQL'S FETCH FIRST X ROWS ONLY command to minimize the number of rows returned
  3. QlikView's DEBUG mode LIMITED LOAD feature

All of the above have their pros and cons.