September 19, 2018

Don’t Let the Completeness and Accuracy of System-Generated Reports Be a ‘Leap of Faith’

I have noticed, much to my chagrin, a lot of CPAs don’t take learning about accounting information systems very seriously.

In college, so many students write it off because “it’s not relevant” and, you know, that belief is only true if you end up working in a large firm and have the luxury of passing off the technical stuff to the IT assurance department. Not everyone's that lucky!

As auditors use data analytics to help perform more audit procedures, it’s much more important to have a working knowledge of information systems. This includes even some of the nitty-gritty stuff like SQL queries. Why, you ask? Because the data we audit comes from relational database management systems and query languages (e.g. SQL) often extract it — even if you can’t see it happening. Just imagine the query is your trusty apprentice behind the curtain (read: a system-generated report) collating the files.

If you're still not convinced that this is important, let me remind you that even programmers mess up. Plus, the propensity for mistakes increases if the programmer doesn’t speak accounting fluently and is confused about the difference between revenue and profit. There is bound to be a glitch here and there. As a result, not every system-generated report is pulling reliable data.

What do I mean by reliable? Well, both system-generated reports and grab-and-go Mexican food must be complete (whole enchilada including the rice and beans) and accurate (fully cooked and assembled to perfection). An enchilada isn’t the same without cheese…

Here’s a list of five risks that could compromise the completeness and accuracy of system-generated reports:

Too good to be true report

Don’t be foolish and think that just because the report is “pretty” means you can trust it! After experiencing my fair share of data extraction horror stories, this is something that shouldn’t be taken lightly. It’s no fun to find an exception in the review phase of an attestation engagement after realizing someone did testing on an incomplete employee list. (It caused a big headache and clients are not happy if you revoke a clean opinion over IT general controls.)

Conclusions — accurate or not — depend on these system-generated reports and if a report is garbage, you guessed it… so is the conclusion. Tommie Singleton writes in the ISACA Journal:

It is tempting to look at a neat report that came from a computer and to have a "leap of faith" as to the veracity and reliability of the information of that report.

It’s always important to make sure the source data and information on the report match and it’s not inadvertently missing a page.

Manager-modified reports

Auditors should keep an eye out for managers who like to tinker. Cars, power tools, model airplanes… oh, and reports. Don’t assume that the vendor was the last one who made a change to the report. Poke around and if you (the auditor) weren’t provided the vendor’s pre-fab report, then someone has some splainin’ to do.

Unexpected parameters and filters

Be on the lookout for query parameters and rogue filters. I spy with my ‘lil eye a WHERE statement! Uh oh… That’s not supposed to be there.

Limiting parameters (like WHERE statements in SQL) keep a query from pulling everything and can mean the data extraction isn’t complete on purpose. For instance, filters should only be used to limit data when there is an auditing purpose like pulling only certain types of transactions or transactions for a specified date.

This has been an emphasis item for the PCAOB in recent years. A 2013 PCAOB report explicitly mentions that auditors failed to “test the logic of the queries (or parameters) used to extract data from the IT applications used in the reports.”

Dancing source data

Data doesn’t hold still and from the time you or your client start a query script to the time it’s finished extracting it is already outdated. It’s really hard to pin down a moving target.

What does this mean for an auditor? It may be hard to verify completeness when the totals are constantly changing. Fear not, auditing friends. There are ways to bypass this hurdle. For example, PwC recommends:

[I]f the underlying data within the report is changing frequently, it may be advisable to have the contact run an ad-hoc iteration of the report to compare to real-time data rather than using a recent version of the report

And if verifying completeness is impossible because the total is changing “it may be necessary to select a specific item to subselect from, or select a sample from a different population. Judgment required.”

Spliced spreadsheets

Beware of the Excel spreadsheets. Extracting data into a spreadsheet is all fine and dandy… until you aren’t there to see it happen. Who knows what rows went “POOF!” and you (the auditor) are none the wiser. It’s so easy to change data after the fact in a spreadsheet without anyone knowing, so it’s important for an auditor to observe the extraction. I will mention, observations of data extractions are not anyone’s favorite task. Clients are nervous that they are going to mess something up and auditors don’t know what they are looking at under the IT system’s hood half the time. But, what’s the alternative? I can’t think of one. Can you?

Isn’t completeness and accuracy fun? Actually, don’t answer that…

Image: iStock/krasnopolski

Related articles

Add Another Hoop to the Audit Process

signature.jpgIn a move that probably just adds one more annoying hoop to jump through for auditors, audit engagements will now go through quality review with adoption of AS No. 7, Engagement Quality Review (EQR).
According to the press release, “The EQR standard provides a framework for the engagement quality reviewer to objectively evaluate the significant judgments made and related conclusions reached by the engagement team in forming an overall conclusion about the engagement.”
We’re hoping the engagement quality reviewers will be given free range to document their “overall conclusions” as they wish. Some that we would suggest: “You call yourselves auditors?“, “I’m recommending that the PCAOB inspect this engagement” or “What in God’s holy name are you blathering about?“. It would be a shame for the firms to institute a check-the-box method that would compromise artistic integrity.
In other PCAOB news, the Board is asking for comments on its Concept Release “to consider the effects of a potential requirement for the engagement partner to sign the audit report.” We speculated last week that signatures in blood or dog excrement might be appropriate in many cases but if you’ve got other ideas, you’ve got 45 days to give them better suggestions.
Press Release [PCAOBUS.org]