Don’t Let the Completeness and Accuracy of System-Generated Reports Be a ‘Leap of Faith’

I have noticed, much to my chagrin, a lot of CPAs don’t take learning about accounting information systems very seriously.

In college, so many students write it off because “it’s not relevant” and, you know, that belief is only true if you end up working in a large firm and have the luxury of passing off the technical stuff to the IT assurance department. Not everyone's that lucky!

As auditors use data analytics to help perform more audit procedures, it’s much more important to have a working knowledge of information systems. This includes even some of the nitty-gritty stuff like SQL queries. Why, you ask? Because the data we audit comes from relational database management systems and query languages (e.g. SQL) often extract it — even if you can’t see it happening. Just imagine the query is your trusty apprentice behind the curtain (read: a system-generated report) collating the files.

If you're still not convinced that this is important, let me remind you that even programmers mess up. Plus, the propensity for mistakes increases if the programmer doesn’t speak accounting fluently and is confused about the difference between revenue and profit. There is bound to be a glitch here and there. As a result, not every system-generated report is pulling reliable data.

What do I mean by reliable? Well, both system-generated reports and grab-and-go Mexican food must be complete (whole enchilada including the rice and beans) and accurate (fully cooked and assembled to perfection). An enchilada isn’t the same without cheese…

Here’s a list of five risks that could compromise the completeness and accuracy of system-generated reports:

Too good to be true report

Don’t be foolish and think that just because the report is “pretty” means you can trust it! After experiencing my fair share of data extraction horror stories, this is something that shouldn’t be taken lightly. It’s no fun to find an exception in the review phase of an attestation engagement after realizing someone did testing on an incomplete employee list. (It caused a big headache and clients are not happy if you revoke a clean opinion over IT general controls.)

Conclusions — accurate or not — depend on these system-generated reports and if a report is garbage, you guessed it… so is the conclusion. Tommie Singleton writes in the ISACA Journal:

It is tempting to look at a neat report that came from a computer and to have a "leap of faith" as to the veracity and reliability of the information of that report.

It’s always important to make sure the source data and information on the report match and it’s not inadvertently missing a page.

Manager-modified reports

Auditors should keep an eye out for managers who like to tinker. Cars, power tools, model airplanes… oh, and reports. Don’t assume that the vendor was the last one who made a change to the report. Poke around and if you (the auditor) weren’t provided the vendor’s pre-fab report, then someone has some splainin’ to do.

Unexpected parameters and filters

Be on the lookout for query parameters and rogue filters. I spy with my ‘lil eye a WHERE statement! Uh oh… That’s not supposed to be there.

Limiting parameters (like WHERE statements in SQL) keep a query from pulling everything and can mean the data extraction isn’t complete on purpose. For instance, filters should only be used to limit data when there is an auditing purpose like pulling only certain types of transactions or transactions for a specified date.

This has been an emphasis item for the PCAOB in recent years. A 2013 PCAOB report explicitly mentions that auditors failed to “test the logic of the queries (or parameters) used to extract data from the IT applications used in the reports.”

Dancing source data

Data doesn’t hold still and from the time you or your client start a query script to the time it’s finished extracting it is already outdated. It’s really hard to pin down a moving target.

What does this mean for an auditor? It may be hard to verify completeness when the totals are constantly changing. Fear not, auditing friends. There are ways to bypass this hurdle. For example, PwC recommends:

[I]f the underlying data within the report is changing frequently, it may be advisable to have the contact run an ad-hoc iteration of the report to compare to real-time data rather than using a recent version of the report

And if verifying completeness is impossible because the total is changing “it may be necessary to select a specific item to subselect from, or select a sample from a different population. Judgment required.”

Spliced spreadsheets

Beware of the Excel spreadsheets. Extracting data into a spreadsheet is all fine and dandy… until you aren’t there to see it happen. Who knows what rows went “POOF!” and you (the auditor) are none the wiser. It’s so easy to change data after the fact in a spreadsheet without anyone knowing, so it’s important for an auditor to observe the extraction. I will mention, observations of data extractions are not anyone’s favorite task. Clients are nervous that they are going to mess something up and auditors don’t know what they are looking at under the IT system’s hood half the time. But, what’s the alternative? I can’t think of one. Can you?

Isn’t completeness and accuracy fun? Actually, don’t answer that…

Image: iStock/krasnopolski

Related articles

Audit Deficiencies Got Much Worse Among Big 4 Firms in Canada

I came across an article on the site Canadian Accountant this morning that said Canada’s audit regulator, the Canadian Public Accountability Board, recently issued its annual inspection results for 2018, finding significant audit deficiencies increased from 15 in 2017 to 34 in 2018. Thirty-four total significant deficiency findings might not seem that bad compared to […]