Download document 8899955

For internal circulation with in IA&AD
PursuIT
IT Audit Wing
The Quarterly IT Audit Journal of Indian audit and Accounts Department
Vol. I
Office of the Comptroller & Auditor General of India
Jan - Mar 2005
Table of Contents
MESSAGE
Wishing You a Very Happy New Year - 2005
1. Crystal Ball
I am pleased to know that the Information
Technology Audit wing has come up with a Quarterly
Journal PursuIT from the first quarter of this year.
•
Alok Ojha, Director (IT Audit)
O/o C&AG of India
2. Challenges in Auditing
e-Governance Initiatives
Information Technology Audit has emerged as one
of the significant areas of focus by Supreme Audit
Institutions the world over. IA&AD also has made rapid
strides in both, use of IT within the department and in
conducting IT Audits of the applications of auditees. IT
environment being a rapidly changing one the IT
Auditors need to keep themselves updated continuously. I
hope that this step taken by IT Audit wing in the form of
PursuIT will help in sharing and disseminating
knowledge in the department.
•
Dr.Ashutosh Sharma, Dy.
Director (IT Audit), O/o CAG
of India, iCISA, Noida.
•
G. Srinivas, Dy. Director,
NAAA, Shimla.
I have gone through the contents of the present
issue of PursuIT and I have found that the issue will
not only enrich the knowledge of the readers on latest
developments in the area of IT Auditing but will also
make them aware about the IT audits conducted in the
department in addition to helping them in solving their
practical problems. I am sure that this journal will
provide the opportunity to learn about something new in
every subsequent issue.
4. Highlights of
3. From Data Analysis
towards Data Mining
•
Rajesh Goel, Sr. DAG
O/o the PAG (Civil audit),
Rajasthan
Information Technology
Audits
5. Let us Try - Importing
ORACLE Data Table
into IDEA
•
K.P. Singh, AAO (IT Audit)
•
S.C. Naithani, Sr. Auditor
With this I congratulate the officials who have
contributed to the journal and convey my best wishes to
the new beginning made by the IT Audit wing.
Feedback
Comptroller & Auditor General of India
[email protected]
[email protected]
Office of the Comptroller and Auditor General of India
International Centre for Information Systems and Audit
i
( CISA), A-52, Sector-62, Institutional Area, NOIDA- 201 301 (UP)
PursuIT
IT Audit Wing
Crystal Ball: A Quantitative Approach
to Risk Analysis
Alok Ojha, Director (IT Audit) O/o C&AG of India
Risk Analysis and Risk Based Auditing are the latest
buzzwords in the field of Audit today. In the modern Risk
based Auditing approach, instead of focusing on risks of
auditors (namely Inherent Risk, Control Risk and Detection
Risk) Audit aligns its efforts with the risks faced by the executive
to increase the relevance of Audit efforts. The ideal goal for risk
analysis is to be completely quantitative i.e. put a monetary value
to all the risks identified by the executive. This is not always
possible and qualitative measures of risk like High, Medium and
Low have to be used in Risk Analysis frameworks.
This article describes a framework of doing a quantitative
risk analysis for Project Management. When planning a project
there are normally two key parameters, the time and the cost.
These are estimates of time and cost. They are expected to vary
because of inherent uncertainties. Measure of these uncertainties
is the risk.
As a simplistic example let us say that there are three
material inputs in the cost of a project A, B, and C. There sum
total goes in the Project Report and becomes a component of
the total cost. If the price or quantity of a material A varies it will
have a final impact on the project cost. If for arguments sake we
assume that cost due to B and C are fixed then the entire
variability (risk) in the project cost will come because of the
variability (risk) in the cost of A. In real life actually all of the
inputs will vary and contribute to the variability (risk) of the
project.
In statistics variance is the measure of variability of an
uncertain variable and therefore if known it can be used to
measure the risk quantitatively. Finding out the combined
variance of the project based on variances of each of these
components though theoretically possible will in practice will be
such complex mathematics that it will be impossible to
comprehend.
Jan - Mar 2005
For scenarios like these quantitative risk analysis tools like
Crystal Ball use simulation techniques to depict the variability of
the final output given the variability of inputs in a graphical
form.
The steps for applying CB to a real life situation are
discussed with reference to an analysis made in “Calculation of
Levelized Tariff for a Power Project” by the Bhakra Beas
Management Board (BBMB)
Description of the Project
In a power project the financial out flow in the initial stages
is more because of initial capital costs, servicing of the loans etc.
Subsequently the costs come down significantly in the case of a
hydro electric project. Therefore a uniform tariff needs to be
worked out taking into account the costs and expenditures over
the time which would assure the required Return on Equity over
a period of time (35 years in this case).
The outgo on account of asset creation in this power
project model is basically costs of civil works and cost of
electrical works. The estimated costs of civil and electrical works
are modelled to escalate every year at an assumed rate. The
financial costs comprise of interest on loan and a working capital
cost. Both these rates are assumed. The discount rate which is to
be used is laid down by an order of Ministry of Power and is
therefore not variable.
BBMB had only done a sensitivity analysis over two
variables and two values in the project appraisal. We expanded
the examination by looking at some more parameters:
Costs
1. Capital Financing Costs
a. Loan Servicing
b. Required Return on Equity
2. Running Expenses
a. Depreciation
b. O&M Costs
c. Working Capital Charges
These parameters and their relations with the final output
was modelled in a spreadsheet by BBMB.
Spread Sheet Model
The project whose risk analysis is to be undertaken should be represented in a spreadsheet model. The model should have the input
components linked to the final components as cell dependencies. The illustration below is helpful.
The existence of such a model is a crucial step in applying CB
Analysis. This is executive’s view of the project. If such a model
is available with the Auditee it obviates the need for creating a
model by Auditor and then getting it validated.
-1-
PursuIT
Jan - Mar 2005
IT Audit Wing
Assumptions
This is the most critical aspect of the quantitative risk analysis
using Crystal Ball. Here we make assumptions about the
“variability” of the input components. In Crystal Ball either one
of the pre-defined statistical distributions or a custom
distribution, where the user defines the probability of the
uncertain variable taking a value (range), can be used. An
understanding of statistical distributions is required for
defensible analysis. These assumptions would be more valid if
they are made on the basis of sound historical analysis.
For example if we assumed that
Assumptions that we made:
(i)
The rate of interest which was taken as 10% and 11%
was taken to vary normally with a mean of 10% and SD of 0.5%.
In simpler terms the probability that the interest rate would be
within 10+/- 1.5% was taken as 99%.
(v)
Energy Availability at presumed 90% efficiency was
taken, we took it to vary normally with SD of 5%
3. Analysis of CB Results
We carried out a simulation for the Levelized tariff with all these
5 assumed parameters varying as per the assumed distribution.
This analysis was done for ROE at 12%, 14% and 16%. The
simulation was carried out with 1000 trials i.e. each of the
variables took a random value as per their distribution 1000
times.
BBMB in its analysis had come to the conclusion that a
Levelized tariff of Rs. 2.76 will achieve its ROE target of 16%.
However we found that there was a nearly 40% probability of
the required Levelized tariff being more than this cut off figure.
This means that there is a substantial probability of the project
not being able to realize the intended benefits. More importantly,
the managements take such risks of not being able to meet the
objectives without realising so.
Furthermore, the sensitivity analysis done by BBMB was with
respect to the interest rates. Crystal Ball sensitivity analysis
showed that the major risk factors lay else where. The risk on
account of Energy Availability and Escalation in Civil works was
significantly more than that on account of interest rate
fluctuations.
4. “Audit Findings” and Analysis
(ii)
O&M Cost were taken as fixed 4% in the model we
took it to vary normally with mean of 4% and S.D. of 0.2%. In
simpler terms the probability that the O&M cost would be
within 4+/- 0.6% was taken as 99%.
(iii)
Escalation on Civil Works cost was taken as 3% in the
BBMB model. We, instead, took it as an extreme distribution
with a mode of 3%, since these escalations are in practise found
to be more than envisaged with a positive skew.
Based on this analysis it can be seen that
(a) The management was taking a risk of about 40% of the
project not being able to achieve its objectives. More
importantly management was taking this risk without even
being aware of it.
(b) The management had not been able to identify and
prioritize the most significant risks it faced with respect to
its project objectives.
CB is basically a decision making tool to aid the management.
When used by Auditors it has the potential of changing the
whole audit approach. The Auditors have long been blamed of
doing a post mortem and there findings being hindsight wisdom
and of little use to the management. An approach based on a
tool like CB can change all this as the findings are futuristic and
can actually help shape the execution of the project. In the
instant case the executive will be able to focus its efforts on
ensuring the power availability which has emerged as the critical
risk factors.
5.
(iv)
Escalation on Electrical Works cost was taken as 1.5%
in the BBMB model. We instead took it as an extreme
distribution with a mode of 3%, since these escalations, also, in
practise are found to be more than envisaged with a positive
skew.
Challenges in Application
The challenges in applying these tools as seen from a few pilot
studies have been the ability to create spreadsheet models and
most critically an understanding of statistical distributions.
Relating statistical distributions to real life scenarios is quite vital
and expertise of a statistician could help.
Û
-1-
PursuIT
IT Audit Wing
Challenges in Auditing e-Governance
Initiatives
“Access to, and flow of, information determines the power
structures in organizations the world over..”
• Dr.Ashutosh Sharma, Dy. Director (IT Audit),
O/o CAG of India, iCISA, Noida,
• G. Srinivas, Dy. Director, NAAA, Shimla.
In the much acclaimed Hindi novel RAAG DARBARI written
by Shrilal Shukla, amongst other things, the travails of a
character named Langad are described in a very humorous but
ironic prose. Langad spends most of the pages available to him
in the novel in the quest of a ‘nakal’ of a ‘dastavez’ i.e. the
copy of a document from the tehsildar’s office, a quest which
remains unfulfilled against an obdurate and all powerful
bureaucracy. E governance is good news to Langad and
millions of his fellow citizens in India.
The INTOSAI Standing Committee on IT Audit Task force
defines e-government as "the online exchange of government
information with, and the delivery of services to, citizens,
businesses and other government agencies". In other words
this encompasses interactions between Government and
citizens (G2C), Government and business (G2B) and inter
government dealing (G2G).
Till date IT Audit wing of SAI India has cleared the IT Audit
reports of 14 e-governance projects, the important ones are
eSEVA, eCOPS (AP) ,Computerization of land records
(Mahrashtra and Tamil Nadu), Computerizations in Municipal
department
Chennai ,Computerization of transport
departments (Delhi and Jharkhand) and Integrated Bus
Reservation System in Maharashtra. This paper is presented
based on the experiences gained from audit of these initiatives.
This paper does not limit itself to what is reported in the
CAG’s report, but covers the entire gamut of issues and
problems faced during the audit
E-government is capable of enhancing the quality of public
services for citizens and businesses by making information
services more accessible (24X7),transparent and convenient to
use. E-governance can enable agencies to process transactions
more accurately and at a lower cost, and can make it easier for
government and non-governmental organisations to share
information with one another. As audit of e-governance
initiatives is technically challenging and a developing area
audit can function as an agent to see that Governments should
make e government part of a wider agenda for change, and not
simply superimpose it on existing, perhaps inefficient,
services.
An e-governance project on the face of it implies removal of
the human interface and substituting it with electronic means
of getting information and facilitating transactions. Use of a
web-based system for dealing with public is the most common
mode, though there are other models also for delivery of
services. However, considering the low level of internet
penetration in India, most of the e-governance projects involve
large-scale use of IT kiosks. These kiosks are the primary
means of e-governance in view of the digital divide in the
country. Eventually most of the projects aim at becoming
completely internet based in the times to come when internet
Jan - Mar 2005
connectivity becomes available to the majority of people. But
till that time the traditional modes of interaction have to
remain operational to serve the public
Audit of e-governance initiatives is not just an audit of
technology used. The scope of audit of e-governance
initiatives by SAI is very wide. It is also the audit of the way
government seeks to work. It involves audit of business
process reengineering. It is also a Value for money audit for
the investment made. At the core of the audit is the IT security
features built into the project. Considering these diverse
aspects, this paper is divided into six broad sections:
1. Business case for e-governance
2. Acquisition process
3. Implementation issues including Quality of service
4. IT Security
5. Data analysis
6. Reporting parameters
1.
Business Case for e-Governance
1.1
Conceptualisation of project: The first question is
whether Information and Communication Technologies (ICT)
are a solution for the problems in governance. E-governance
projects are by nature huge and complex. Such a decision
needs to be taken on careful analysis of cost and benefits of
the project. The auditors challenge in this area is due to the
fact that the decision making process is not documented in
most of the cases and inferences may have to be drawn from
actual results of implementation of project.
1.2
Feasibility Study: Auditor faces with the dilemma as
to whether he should insist on a well-documented feasibility
study, User Requirements and System Requirement
Specifications. As IT is in itself at a nascent stage, it is quite
likely that the auditor faces with a situation where no such
documents are available. (The Government replied to an audit
query in case of eSeva stating that had they kept working on a
detailed feasibility study, the project would have never seen
light of the day). In such a situation, it would be reasonable to
depend on files and other documents to understand whether
adequate thinking has gone into before taking the decisions. In
all probability most of the problems in development and
implementation would be a direct consequence of inadequate
feasibility study etc and would become basis for comments on
inadequate planning and evaluation.
1.3
Cost Benefit Analysis: Audit faces with a problem of
evaluating whether the costs of the project justify the benefits.
Firstly, identifying the costs would be an issue in a situation
where a private partner gets transaction charges from the
government. Secondly, it is very difficult to quantify benefits
many of which are like increased transparency, convenience,
etc. The auditor has to rely on reasonability in making
judgment about benefits. Parameters like quicker service
needs to be tested based on examination of recorded average
time before and after implementation of project. If the
Government got prepared a Cost Benefit analysis report like in
the form of Pay Back Period or Break Even Point etc (eCops
project planning documents included a statement showing Pay
Back Period based on savings from reduced manpower), the
same should be tested for reasonability. Savings in form of
diversion of surplus staff due to e-governance to other projects
needs to be examined. In most of the cases one would find that
-2-
PursuIT
the staff strength remains the same even after new technology
and systems have come in place.
1.4
Case for outsourcing: The recent trend in
Government to outsource IT activities to private sector needs
attention of auditor. It would be reasonable to rely on
judgment of the executive regarding whether to go in for
outsourcing and the form in which outsourcing is to be done
viz., software development outsourcing, BOOT basis or public
private partner-ship. The auditor should make sure that well
informed decision is made after detailed analysis of
alternatives. The auditor should make sure that sovereign
duties or statutory functions of Government are not
outsourced. For example, it would be unreasonable to
outsource the responsibility to maintain data relating to FIRs
etc to a private party.
1.5
Identification of services to be included in egovernance: The guiding principle should be what the people
want to access on-line. The auditor may recommend additional
areas that may come to his notice during audit for inclusion.
Here auditor can work as a change manager to encourage
widespread use of e-governance projects.
1.6
Business process reengineering: Introduction of egovernance involves not merely change in technology, but
may also require reengineering the business process. This
involves redefining the roles and responsibilities of various
functionaries. The auditor should evaluate the adequacy of
changes in the process to facilitate and take benefits of egovernance.
2.
Jan - Mar 2005
IT Audit Wing
Acquisition Process
2.1 Preparedness of various departments/functionaries: This
is one of the most important prerequisites for implementation
of an e-governance project. In most of the projects, none of the
functionaries/departments are found ready. The databases are
not compatible, the hardware and software are not compatible
with existing ones. These lead to problems in implementation
and eventually increased cost and delays.
2.2 Selection of Vendor of hardware/software: The selection
of vendor should involve careful examination of capability of
the firm. This particularly important if customized software is
to be developed by the vendor. The audit can bench mark the
acquisition process using models like Software Acquisition –
Capability Maturity Model (SA-CMM).
2.3 Use of frameworks in software development and project
implementation: The auditor is expected to examine whether
the project management has used any of the established
frameworks like CoBIT, CMMi etc. If so, auditor can examine
the extent of compliance with framework. Even if the project
management has not adopted any specific framework, the
auditor can rely on good practices of these frameworks while
making examination. We often relied on CoBIT audit
guidelines for use as Half Margin memos in conduct of audits.
The problem faced is that these have to be amended to suit the
audit goals in the situation. Initially when we issued
questionnaires as it is, the auditee did not even understand
how to reply and the replies received did not even reflect
actual views of project management. In many cases, the
replies were found to be self-contradictory or conflicting with
each other. We then carefully amended the questionnaires
based on what assurance we were trying to achieve in audit
and explained them in detail to the auditee and to get proper
response.
2.4 Development of Software: The software methodology
should involve an established good practice like System
Development Life Cycle (SDLC). There should be a right to
audit the development firm’s work by the acquirer. In the
absence of such clause and its effective enforcement, the
auditor should seek alternative methods of obtaining
satisfaction and may also need to qualify the report. In case of
eSeva, the entire development of software was left completely
to private firm with no role from Government, which
eventually resulted in avoidable problems in the software.
2.5 Contract document: The contract should be examined
from legal perspective to make clear the responsibilities of
both parties including incorporation of confidentiality clause.
There should be adequate penal provisions on the vendor for
poor quality of programme, cost and time overruns, etc. Often
it is noticed that though many penal provisions are present in
the contract, most of them are never invoked giving
unintended benefit to the developer.
2.6 Maintenance contract: It should be examined in audit
whether adequate provisions for post implementation
maintenance of hardware and software are provided. In case
the maintenance is to be taken up in house or by some other
vendor the capability should be examined.
2.7 Acceptance testing: The auditor should ensure that project
(application) is finally accepted after detailed stage wise
acceptance testing by various user groups. Inadequate
acceptance testing results in many problems at a later stage.
2.8 Need for physical verification: The IT auditor may on
many occasions need to physically verify the hardware. For
this may have to visit various locations. It was seen in one of
the audits that, on physical verification, private partner has not
complied with contractual provision to a very significant
extent regarding providing hardware and facilities at service
centers. We ensured that a designated official from auditee is
always present during audit. Physical verification may also be
necessary to evaluate the adequacy of physical and logical
access controls. IT audit on many occasions results in findings
which exist at a particular moment and no document of
evidence except physical inspection by auditor is possible. In
such cases, auditor should rely on counter signature of official
of the auditee present at the location as acceptable audit
evidence.
3.
Implementation Issues Including Quality of
Service
3.1 Web-based services: These should be in such a way that
citizen is encouraged to access web services rather than
seeking services through traditional methods of visiting office.
While kiosks are a transitory process, real cost savings and
convenience can only come through website based services.
The website should be designed in such a way that it provides
all information to the user regarding access to services. It
should be user friendly. It should give an overview of security
like the Digital Certification, payment gateway etc to increase
the confidence of the user. It should be continuously updated.
The initial response to web-based services may not be very
encouraging due to the public concerns about security in using
internet. However, this should not be a reason for not keeping
everything accurate and updated on the net. The popularity of
web based services increases if the initial visitors to the site
-3-
PursuIT
IT Audit Wing
get good service. Web services should not be launched
without adequate processes and quality services in place from
day one. In many cases, the website and services were in place
from beginning, but the sites were neither updated nor
convenient to use. Lot of procedural requirements including
manual intervention by way of follow up letters etc was
involved. Eventually when everything was set right, it took
time for people to come back and use the web services.
3.2 Provision for paper based receipt in case of web
payments: There should be a provision for the user to print a
paper receipt as an evidence of making the payment. The
auditor should verify the adequacy of the same.
3.3 Quality of service: The auditor needs to examine the
quality of services under e-governance. This is the most
important facet of audit of e-governance projects. The
parameters are wide and varied. Broadly these include
parameters indicating efficiency of project and user
orientation. Efficiency can be gauzed from speed, security,
superiority over traditional methods, etc. User orientation
involves examining how easy is it to access services, extent of
clubbing services at single location, user support and problem
resolution, local language interface, friendly and supporting
approach of staff in kiosks, reduction of need to visit
government offices repeatedly etc. It was a major challenge to
evaluate projects like eSeva and eCops on these parameters. In
case of eSeva we relied on a survey conducted by Government
using some research and educational institutes. We also used a
questionnaire to collect information from citizens present at
eSeva centers to supplement the work done by Government
nominated institutes. In case of eCops, the problem was even
more complex due to huge resistance within the auditee
department to the project. Even in cases where eCops provides
lots of convenience, the departmental users were found to
paint a grim picture to tarnish the image of the project. The
response of people could not be tested in case of eCops due to
administrative constraints.
4.
IT Security
4.1 Adequate physical and logical access controls: These
need to be examined keeping in view the risks. This includes
all the basic checks to be performed regarding general controls
and application controls (Input, output and processing). Most
significant part of our time of audit was spent on this area
since there were many controls that were absent or
insufficient. The data in these projects could be amended at
any stage by multiple sources.
4.2 Encryption of data: Considering the fact that huge and
important data travels over the network or internet, it is
essential that data is encrypted. Most of the time auditor finds
himself in a dilemma when project management replies that
encryption has not been resorted to, as it would slow down the
system. In IBRS Maharastra there was no token based
authentication system to identify authorized ticketing agents
which would have served some purpose of compensating
controls. The auditor should take into account the risks and
established compensating controls before forming an opinion.
4.3 Use of PKI: Public Key Infrastructure and use of Digital
Signatures has legal validity under IT Act 2000. Auditor
should evaluate the extent PKI has been adopted depending on
the nature of the project
Jan - Mar 2005
4.4 Penetration testing of website: Ethical hacking or
penetration testing has been one of the accepted practices of
IT audit. The auditors should decide on these tests taking into
account their own competence and whether any such tests are
conducted by project management. In all cases, the auditor
should take prior approval of the auditee before conducting
any such tests.
4.5 Source code: The auditor should see whether the package
used open source system. The auditor should ensure that
source code of the package is available with responsible
government official. Auditor may if necessary run a code
comparison programme to compare the executable with that in
the programme library. This gives the auditor an assurance
that no unauthorized programme is running on the system. In
case of eSeva it was noticed that private operator bluntly
refused to share with source code and government insisted and
got the same at the instance of audit
4.6 Segregation of duties: This is one of the significant areas
which is likely to be found lacking. Auditor should not merely
check for clubbing of incompatible functions (database
administrator and systems administrator) but should see it in
overall context in relation to existence of any compensating
controls before forming an opinion.
4.7 Protection against viruses, worms etc: Auditor should
examine the project for adequacy of protection against Trojan
horses, viruses, worms etc. He should also test the website for
possibility of Denial of Service attacks. A successful DoS
attack on a well established e-governance project leads to not
only inconvenience to citizens but also severely affects the
public confidence in the system.
4.8 BCP and DRP: Since most of e-governance projects are
critical and huge in nature, the plans for Business Continuity
and Disaster Recovery should be reviewed by the auditors.
Auditor should also examine the arrangements of Hot site,
Warm site or Cold site as part of BCP. In IBRS Maharashtra
entire data beyond a certain date was lost and could not be
recovered due to absence of backup.
4.9 Use of outside expertise: Since SAI may sometimes be
lacking in core technical skills like conducting network
testing, website ethical hacking (penetration testing), testing
firewall configuration etc, it is accepted practice to use outside
experts. However, care should be taken to ensure that the
outside expert appointed does not misuse his access to data
and system. Proper confidentiality clause should be
incorporated in the contract with the outside expert for this
purpose. During the audit of eSeva, we tried to use the
services of NIC for conducting some core technical network
testing. The Government in meetings expressed serious
reservations about using outside experts. Taking into account
their objections and our own perceived necessity, we
abandoned the idea and conducted the network testing
ourselves.
4.10 Program change controls: This is an important area of
concern for auditor particularly in a project of e-governance.
Most of the time, it may be noticed that program changes are
made based on discussions of programmers with users without
any documented approval procedure. This can lead to serious
risk of unauthorized program changes.
4.11 Security over monetary transactions: In e-governance
initiatives involving eCommerce, security over monetary
transactions becomes very important. Auditor should ensure
-4-
PursuIT
that adequate control exists to account for the collection of
money, prevent misuse of credit card information, etc.
5.
Data Analysis
5.1 Use of audit tools: Auditor may use tools like IDEA,
Excel, SQL, Access etc to analyse the data. When egovernance integrates data between different departments etc,
data transfer and integrity are of paramount importance. These
tools can throw light on serious limitations in the database.
Based on data analysis auditor can form an opinion as to
whether the information generated from the e-governance
project is dependable. Data analysis also helps auditor in
forming opinion about various application controls like input
controls, processing controls, output controls and edit checks.
Auditor should be cautious to ensure that data is properly
copied into IDEA etc package so that the results are accurate.
In eSeva, eCops and IBRS audit, many of our audit results
were collaborated by data analysis using IDEA and SQL.
5.2 Use of embedded audit module: In case the auditors is
associated at the stage of developed itself, they can ask for an
embedded audit module which copies exceptional data to a
separate audit file for review by auditor. Audit should also
insist on adequate audit trails to be built in the application at
the developmental stage.
6.
Jan - Mar 2005
IT Audit Wing
Reporting Audit Findings
6.1
Balanced reporting: Use of e-governance leads to
increased transparency and helps further participatory
democracy. Most of these projects have the capability of
altering the power base in favour of people. These help in
curbing corruption and misuse of power by executive. These
will always be many individuals who would like to see that
these projects fail as they threaten the existing power
structures. Auditor should be cautious to ensure that audit
reports do not result in giving means for unscrupulous
elements to kill the project. The reports should be balanced
with recommendations. They should also reflect the efforts
and success of government in implementing the projects. The
page limitation should not be a reason for printing a report
which is completely one-sided. While performing IT audit of
e-governance projects, the role of SAI is not only to bring
lacunae to notice of legislature, but also to help project
management improve the project.
6.2
Impact of individual observations on overall project:
Auditor should take into account the impact of individual
weaknesses on the overall project. Auditor need not report
weaknesses for which adequate compensating controls exists.
The report should clearly indicate the impact on overall
project so that legislature / executive can decide on the need
for corrective action.
Conclusion
e-Governance is most of all about people, it is an
enabler of change rather than an end in itself. On the issue of
e-governance it is often seen that it is not the government
which sets the agenda but the enlightened citizenry and
credible organizations have an important role to play. SAI
India can play an important part in this process by auditing
these e-governance initiatives in a constructive manner. This
will not only help in assuring the legislature about the
usefulness of these initiatives but also lend credibility and
create trust amongst citizens for these services. E-governance
seeks to completely change the way we perceive the
government. This assumes importance especially with the
Right to Information Bill soon becoming a reality. Here the
role of audit is to give an opinion about processes ensuring the
security of sensitive data, and creating trust among citizens
and businesses that may be understandably nervous about
transacting business on-line. Individual technical objections
when reported by media based on CAG’s report could be
blown out of context and proportion. Development of egovernance itself is at a very nascent stage in the country. Our
reports should therefore ensure that they are balanced with
sufficient coverage of achievements of the project
management. Extremely critical reporting from SAI without
coverage of positive achievements would hamper the
development of e-governance. The baby should not be thrown
with the bathwater. Therefore, auditor should ensure that the
reporting is balanced and constructive.
“The future is electronic and we better be prepared for it”.
Û
From Data Analysis … towards Data
Mining Audit
Rajesh Goel, Sr. DAG, O/o PAG(Civil Audit),Rajasthan.
Data Analysis had brought out some very emphatic audit
findings in IAAD’s IT Audit efforts re-emphasizing the audit
value that data analysis has. Most of the published IT Audit
reports are banking heavily on conclusions drawn as a result
of data analysis carried out by auditors with the help of data
analysis software. As per Strategic IT Audit plan of 2003, four
kinds of generalized audit software or CAATs were identified
for usage in the department namely MS Excel, MS Access,
IDEA and Structured Query Language. Performing audits
without using such IT tools is hardly an option. When all the
information needed for doing an audit is on computer systems,
how can one carry out an audit without using the computer?
Much has been discussed and said about the data analysis
software and tools adopted by IAAD, so this article will not
address them. However, the subject of data mining is
relatively new and the focus of this article would be to
introduce data mining which is a step ahead of data analysis.
Data mining refers to extracting or mining knowledge from
large amounts of data. There are data mining tools available
for auditors to extract information carried in large databases in
an easily comprehensible form. For the purpose of
understanding and appreciating the added features over and
above data analysis tools, a case study depicting the use of a
data mining tool would be presented in this article. The data
mining tool that has been chosen for the purpose of this article
is WizRule1.
1
WizRule is a data mining tool from WizSoft suite from Wiz Soft Inc. The
price of the software is approximately Rs. 80,000 per license. WizRule can
handle a large number of data bases directly like Oracle, MS Access and
many others using ODBC. A demo copy of the WizRule software can be
downloaded from the website www.wizsoft.com
-5-
PursuIT
IT Audit Wing
WizRule is a data-auditing tool based on data mining
technology. WizRule performs a complex analysis of data
revealing inconsistencies, errors and cases to be audited.
Almost anyone who works with data – from end users to
database managers – is well aware of the great number of
errors that occur in data. These errors are the result of a range
of different factors. In many cases, they are caused by faulty
data entry, whereby the user types in one value instead of
another. In other cases, errors are made intentionally, such as
in cases of fraud. Errors are sometimes also the result of
software or hardware malfunctions, resulting in corrupted
data.
The WizRule program implements an innovative approach to
automatic data-auditing. It is based on the following
assumption: In many cases, errors are exceptions to the
norm. For example, if, in all sale transactions to a certain
customer, the salesperson is Dan, a single transaction in which
the salesperson is someone else, who is usually connected
with other customers, can be considered a “deviating
transaction” or a suspected error.
In creating a software application that discovers exceptions to
the norm, the program first needs to discover all the rules in a
given data set. This is precisely Wiz Rule’s strong point.
WizRule is based on a mathematical algorithm that is capable
of revealing all the rules governing a data set within a very
short span of time. The output of a WizRule analysis is a list of
records that are unlikely in reference to the discovered rules.
These records are suspected errors, or at least cases to be
examined.
Although WizRule is based on sophisticated mathematical
algorithms, the software has been designed for users with little
or no knowledge of mathematics. WizRule performs its
calculations in the background and then displays the results of
the analysis in clear, easy-to-understand formats. We need
only select the file to be analyzed, and WizRule does the rest
of the work.
WizRule examines data seeking to understand relationships or
rules that may exist. The rules may be of the if-then type or
they may be formulas. An example of an if-then rule is:
If Customer is Apex and
Order Item is Scanner
Then
Salesperson = John Goodside
Rule’s probability is 0.98
The rule exists in 103 records
Deviations (records’ serial numbers):
11,54
An example of a formula rule is:
A=5*B
Where: A = Value of a stock
B = Net Profit
Rule’s accuracy level: 0.99
The rule exists in 1890 records
Jan - Mar 2005
Deviations (records’ serial numbers):
43,378,453,567,568,789,800,904,1010,
1800
WizRule allows the auditor to adjust the minimum probability
level of the if-then rules, the minimum accuracy level of the
formula rules, the minimum number of cases of a rule and the
maximum number of conditions of a rule. Here is a real-world
case.
Embassy Vouchers’ Data Analysis
For the purpose of evaluation, WizRule was applied to a
database of approximately 25,000 vouchers of embassies. This
database in MS Access was developed at Headquarters office
in the year 2001 and contained the information about vouchers
generated in five major Indian missions. WizRule, since it
discovers all the rules that exist in the data helped us in
identifying the business rules that govern the data and also
point out the departures from these rules. The departure from
these rules would indicate errors in system application
controls, errors in data entry and even frauds. In either case
they would identify cases for detailed audit.
Application of WizRule to the database of vouchers of selected
embassies generated a Rule Report having more than hundred
rules in order of significance. A casual browsing shows that
these rules in many cases confirm the existence of business
rules. This is helpful to an auditor to draw assurance about the
system. The deviations are nearly always audit findings or
certainly cases for further investigations. A few illustrative
cases are listed below:
Case I
Visa Fee should always be a receipt therefore cases where
there is expenditure on this object head are certainly cases of
erroneous data. A further analysis can be carried out to
examine these individual cases.
If objectExpenseName is Visa Fee
Then
expOrReceipt is R
Rule's probability: 0.954
The rule exists in 208 records.
Significance Level: Error probability is almost 0
Deviations (records' serial numbers):
2198, 8349, 8396, 16648, 16664, 16665, 16667, 16668,
16669, 22979
Case II
In London Embassy 97% of the vouchers are expenditure
vouchers. The data could be incomplete or point to a
peculiarity in London Embassy.
If missionName is London
Then
expOrReceipt is E
Rule's probability: 0.973
The rule exists in 8260 records.
Significance Level: Error probability is almost 0
Deviations (records' serial numbers):
1795, 1796, 1798, 1799, ……
-6-
PursuIT
IT Audit Wing
Conclusion
The data analysis by tools like IDEA require the auditors to
first develop the audit criterion and then carry out the analysis
against this criterion. Developing an audit criterion is a
process which is normally very much dependent upon the
auditor’s knowledge about the entity, skills in risk assessment
etc. Traditionally auditors used to look for isolated cases of
irregularity and it was their skill of zeroing in on such cases
that made them become better transaction auditors. In fact it
can be argued that even in the wake of developments like
application of statistical sampling etc. the true skill of an
auditor still lies in identifying a “biased” sample, which would
contain most of the irregularities. As is clear from the analysis
above, a data mining tool can help us on both the fronts. It can
develop the audit criterion, throw up a sample to be
investigated and give assurance about the remaining data. The
deviation report from the WizRule gave us a set of transactions
and also tells us what to look for when auditing these
transactions. For example the deviation report regarding
classification of above data tells us:
1. What are the vouchers where classification is wrong?
2. Check these transactions to ascertain why the
classification was wrong. Whether it was due to
a. Errors in data entry
b. Erroneous data processing
c. Deliberate error of commission
3. Gives us assurance that the classification is correct in
the remaining vouchers.
Thus a data mining tools has high applicability in the
following areas:
(a) Establishing the existence of business rules and thus
establish data integrity.
(b) Drawing assurance about controls on data entry.
(c) Drawing a sample for audit which fulfils audit
criterion and helps in zeroing in on potential audit
findings and even unearthing frauds.
Û
Jan - Mar 2005
Maharashtra
Industrial
Development
Corporation
(Corporation) had computerised its water billing system with
Oracle as a back end and Developer 2000 as front end to
generate water bills and to maintain database of its consumers.
A review of the water billing system revealed that as per the
instructions issued (November 1997) by the Corporation in
this regard, the industrial consumers were to be charged 50 per
cent above the normal industrial rate if they had not obtained
Building Completion Certificate (BCC) on or before 1
December 1997. The database for the water billing system had
two critical fields, which controlled the billing for BCC
purposes; 'BCC date' and 'BCC field', both with 'Yes/No'
option. The water billing system was programmed in such a
way that if the 'BCC field' was 'Yes' (i.e. BCC submitted) the
consumer would be billed at normal rate and if the 'BCC field'
was 'No' (i.e. BCC not submitted) then the consumer would be
billed at 1.50 times the normal rates. However, the
Corporation had not fed the 'BCC date field' in billing system
and it was left blank. This was a crucial lacuna as the date of
obtaining the BCC was crucial for further billing purpose.
(b) Non incorporation of validity checks and improper coding
in water billing system resulted in loss of revenue of
Rs.2..04 crore.
In the absence of validation check for linking the 'BCC date
field' and 'BCC field' with 'Yes/No' option, 106 consumers
who had not obtained 'BCC' on or before 1 December 1997
had been billed at normal rate instead of higher rate resulting
in a revenue loss of Rs.1.70 core in respect of Thane,
Dombivali and Ambernath divisions of the Corporation during
the period December 1997 to March 2001. In respect of
Ambernath division, Rs.34.37 lakh collected by way of
penalty for non production of BCC from consumers had been
subsequently refunded wrongly (August 1998) to the
consumers by considering the date of starting production as
‘BCC’.
Thus, non incorporation of suitable preventive, detective and
corrective validation checks and improper coding of
parameters of business rules resulted in loss of revenue of
Rs.2.04 crore due to under billing of water charges.
The Corporation stated (December 2002) that the irregularities
mentioned in Audit para were examined and modification
carried out in the water billing software. Efforts are being
made to recover the amount due to the Corporation. The reply
of the Government was awaited (December 2002).
Highlights of some of the Information
Technology Audit reviews which have
been printed.
1. Maharashtra Industrial Development Corporation
-Under billing of water charges
(a) Non incorporation of validation checks and improper
coding in water billing system resulted in under billing of
Rs.2.04 crore .
2. Computerisation of the activities relating to
Housing Department of Delhi Development
Authority
The computerised database of DDA relating to allotment of
flats and realisation of dues from allottees was incomplete,
inaccurate and unreliable.
No input controls were provided to keep a check over the
allotment of flats to ineligible minors and also over the double
-7-
PursuIT
Jan - Mar 2005
IT Audit Wing
allotment of flats to the same individual in contravention of
the guidelines.
Lack of monitoring of cancelled flats led to delay up to 10
years in their re-allotment as per the computerised database.
The database was unreliable and depicted an untrue picture of
the dues recoverable by DDA mainly due to non updation of
the cancelled flats in the allottees’ database leading to
duplication of demands. As per computerised database, in
19826 cases of allotments, not even a single instalment was
paid by the allottees up to March 2000 against the demand of
Rs 310.47 crore and in 46893 other cases, where the flats were
allotted on cash down basis, no payment was made by the
allottees against the demand of Rs 697.94 crore.
Injudicious termination of a contract resulted in avoidable
delay of more than two years in implementation of an online
Housing Management Information System besides extra
avoidable expenditure of Rs 19.88 lakh.
Failure to take over operation of Housing Management
Information System named 'AWAAS' completed by CMC
Ltd. in October 1999 and accepted by DDA in November
1999 resulted in expenditure of Rs 32.88 lakh till October
2002 on its operation by outsourcing against the cost of Rs
43.25 lakh for application software development.
'AWAAS' Application was not fully operational despite Rs
1.95 crore were spent on its development including the cost of
Hardware, System Software and Networking.
3. Information Technology Audit of eSeva – an
e-Governance initiative by Government
Though Government launched a unique and
conceptually a good project to put e-governance into action to
provide a large number of services to citizens on one-stopshop basis, the project suffered from lack of transparency,
inefficient and ineffective implementation largely due to
unpreparedness of the participating departments and
inadequate coordination. The network was exposed to serious
risks of physical access controls and logical controls. The key
data and huge volumes of cash pertaining to various
departments had been left to the administration of private
operator without adequate internal controls. Data integrity,
reliability, and safety across the project were also inadequate.
The eSeva project, a New Service, was started without formal
budget provision and without conducting feasibility study.
Financial rules were largely neglected by the Director, eSeva
project in implementing the programme. The project was
rushed through implementation even when the participating
departments were not ready.
The bid evaluation adopted in selecting the operator lacked
transparency, and only one operator was selected instead of
two in violation of the Government orders.
Adequate documentation did not exist for any of the aspects
relating to software, hardware, network, error handling, etc.
Complete technical documentation including the source code
specified in the tender was also not obtained. This had resulted
in a situation where the Director was completely dependent on
the operator. Adequate business continuity plan also did not
exist.
The essential controls in computerised environment such as
logical access controls, physical access controls, etc. were
found inadequate. The network security of the project was also
lacking.
The transactions in eSeva were not reconciled with the data in
the respective departments and scrutiny revealed many
irregularities, inadequacies and inconsistencies in the data.
Government assets worth Rs 90 lakh relating to the TWINS
pilot project were handed over to the operator free of cost
though not provided in the agreement.
4. Information Technology Audit review on the High
Tension Billing System of Maharashtra State
Electricity Board, Mumbai.
The computerised high tension (HT) billing system of
Maharashtra State Electricity Board (Board) was initially
implemented in 1981 and re-engineered during 1997-2000.
Considering that about 58 per cent of the total revenue is
generated from HT consumers, the system handling HT billing
and revenue realisation is ‘mission critical’ in nature.
In the absence of a formal information technology (IT) policy
and long term strategy, the IT center sites prepared during
April 1999 to August 2002 at a cost of Rs.1.40 crore were not
made operational due to delay in procurement of hardware.
The Board incurred expenditure of Rs.1.54 crore on
outsourcing of billing due to delayed commissioning of IT
centre at Bhandup.
No policy regarding physical and logical security of IT assets
including software and data existed. Insufficient security
features with respect to access control, passwords and login
control rendered the system vulnerable to unauthorized access
and data manipulation.
The disaster recovery and business continuity plan was not
documented. The data backup was not periodically checked to
ensure recovery of data.
In the absence of undertaking by Price Waterhouse Associates
for passing on intellectual property rights to the Board, the
system design, source codes of IT billing system developed
are vulnerable to misuse.
There was waiver of minimum charges of Rs.7.13 crore and
non levy of charges of Rs.1.54 crore in violation of rules.
Delay in issue of bills to HT consumers (Rs. 868.44 crore)
resulted in loss of interest of Rs. 1.15 crore.
Excess bulk discount of Rs.3.19 crore was granted to
ineligible HT consumers and incorrect calculation of power
factor incentive resulted in excess rebate of Rs.5.58crore.
Û
-8-
PursuIT
Jan - Mar 2005
IT Audit Wing
Let us Try
Importing ORACLE Data Table into IDEA
K.P. Singh, AAO (IT Audit)
S.C. Naithani, Sr. Auditor(IT Audit)
Since the inception of the IT Audit Wing at Headquarters, one of the major problems raised by the field representatives of different
audit offices is how to download the oracle data into IDEA (Generalized Audit Software) adopted as one of the CAATs in the
IA&AD. Although, the ORACLE data can be very well queried using SQL Commands, IDEA being very popular and User Friendly
Interrogation Software, the present article is an attempt to help and guide to all those who wish to use IDEA as the main
interrogation tool.
The detailed procedure for downloading Oracle Data into IDEA using ODBC driver is explained in the subsequent paragraphs:
Ensure that the PC in use on which IDEA has been installed
is connected to the auditees Data Base Server (in other
words it allows you to Log on to their System.
Diagram – 1
Open the IDEA Software
Select the Import Assistant
Click on Use the ODBC to import the file option and press
next as indicated in the Diagram – 1.
ODBC Import Box appears on the screen
Diagram – 2
A list of available ODBC Drivers is displayed in the Box as
indicated in Diagram – 2.
Press on Create an ODBC Data Source button.
Pressing on Create on ODBC Data Source will open the
screen as per Diagram – 3
Diagram – 3
Enter the DSN (Data Source Name) in the space provided.
Click on OK
If there is no connectivity between your pc and the Data
Source Name, an error message will be displayed. Else it
will take you to next screen as shown in Diagram 4.
Once the Screen as shown opposite is displayed, select the
Oracle ODBC Driver option from the list of drivers
Diagram 4
Press Next
if there is no connectivity, this will return an error message
“unable to access file”.
Else this prompts you to the next screen as per Diagram 5.
-9-
PursuIT
Jan - Mar 2005
IT Audit Wing
The Diagram 5 asks you to enter the file data source you
want to save this connection to. Or, you can customize the
location by clicking browse button.
Diagram 5
Press Next
This will take us to the next screen as shown in Diagram 6.
Diagram 6
Create New Data Source box appears and indicates the
details of File Data Source (Filename and Driver name)
Press Finish button.
Enter the User Name, Password and the Server Name in
the Microsoft ODBC for Oracle Connect box.
Diagram 7
(Note: the User Name, Password and Server Name to
make use of the auditees database while Importing the
tables into IDEA is to be obtained from the DBA of the
Auditee organization)
Pressing the OK button will end the process of creation of
an ODBC Data Source. In subsequent steps this created
data source will be utilized for importing ORACLE Data
Table in to IDEA.
Once the creation of ODBC Data Source is done this can
be used any number of times for Importing the ORACLE
Data Tables from the same server. For this:
Diagram 8
Open the Select Data Source screen (Diagram 8).
This will show the name of DSN name created by us.
Select the created DSN name and press OK.
- 10 -
PursuIT
Jan - Mar 2005
IT Audit Wing
Diagram 9
This will open the Microsoft ODBC for ORACLE Connect
box (Diagram 9).
Enter the User Name, Password and the Server Name
Press OK
Diagram 10
Next screen (Diagram 10) will list the Available Tables in
the auditee server as shown in Diagram 10. Select the
desired table(s).
The button Check Size of ODBC Import will ascertain the
disk space required for importing the selected table(s).
Press Next button.
Diagram 11
This brings us to the last screen (Import Assistant –
Specify Idea Filename) of importing the table.
Enter the Idea file name for the imported table in the Name
of Database
Press Finish
Diagram 12
After importing the table, the imported table(s) can be used
as other IDEA files. Analysis, Extractions, Sampling
Techniques can be made use of. The imported table will
appear like one shown in the Diagram 12.
Û
- 11 -