Download BI Rough Draft

What is Business Intelligence?
Business Intelligence or BI, is a rapidly advancing technology-driven process used for
data analysis. The tools and skills involved compile data to deliver actionable information to
managers and executives. Some examples of these tools are SSAS Tabular, Power BI Desktop,
and Power Pivot. SSAS Tabular and Power Pivot are similar in that they compile data and allow
users to write calculations (measures), on vast amounts of information. Desktop applications
compile data in a similar way except with emphasis on visualizations. The benefits realized from
these technologies are faster decision making, optimized internal business processes, and
increased efficiency. These benefits allow companies to realize competitive advantages. Firms
that are adopting these tools are realizing these advantages quickly. Even these technologies
are only the bare bones of what the future holds. BI technology is progressing at an astounding
rate and if it hasn’t yet, it will impact the way you do business. Though we have only scraped
the surface, the reader will soon realize that these new BI technologies are the beginning of the
data revolution! Companies that learn to use their data effectively will win in the market, those
who do not will fail.
The term Business Intelligence first appeared in the 1860’s. It was not used in the
current context however, until 1989 when Howard Dresner proposed the term as an umbrella
category. He defined it as a set of data analysis techniques that support business decisionmaking processes. BI technologies evolved from executive information systems and decision
support systems. Others have defined it as a set of “computer-based techniques used in
identifying, extracting, and analyzing business data – such as sales revenue by products and/or
departments, or by associated costs and incomes.” Most recently its use has been extended to
business users and IT decision makers equally. Its purpose is to provide historical, current and
predictive views of a business.
Business Intelligence is most commonly used for data mining, analytics, reporting,
process mining, benchmarking, performance management and more. The current
understanding of this technology began in the early 1990s with enterprise resource planning,
which aimed to link corporate databases with client/server technologies. The goal was to
support the selling process, finance, human resources, and other aspects of business. The rise
of the internet allowed these technologies to reach heights never before imagined. It wasn’t
until 1997 that the term “Business Intelligence” became widely used. That year forecasters
began to widely adopt these ideas and leverage them to perform predictive analyses. The need
for these technologies was underscored in 1994 when news about Wal-Mart reported that its
headquarters housed 460 terabytes of data, or more than double the amount of the internet at
that time! By 2010 67% of “best in class” companies boasted some form of self-service BI.
Current forecasts show that the use of business intelligence will continue to grow and evolve.
This means that it is becoming increasingly important for business users to learn, embrace, and
implement these technologies into their day-to-day activities.
The power of these technologies has evolved to a point where they are accessible to
even the smallest of businesses. Business Intelligence was once reserved for large companies
with vast amounts of capital and technological capability. Today a single user can access the
same capabilities at their desktop with services from Hadoop, Oracle, and Amazon. Self-service
analytics are being used in ways that were not possible ten years ago. This vast availability and
ease of access is what is defining the data revolution.
Data storage/DSR
BI has changed significantly in the past 10 years. Analysts and IT professionals are
gaining access to something called a Demand Signal Repositories. To understand what these
are, the user must first understand what a demand signal is. Some traditional demand signals
are consumption and inventory shipments and stock levels. Consumption is a type of
downstream data source and describes the amount being purchased at a given time.
Downstream is a term used to describe the data that we receive from the end of the value
chain. Conversely, upstream data is the data that we are receiving from our suppliers,
manufacturing plants, and possibly warehouses; depending on where we are viewing the value
stream from. Inventory and shipment stock levels are the amounts of product that are being
shipped or are currently available in store or in the warehouse.
The evolution of technology is providing us access and understanding of another type of
demand signal, the unstructured form. This type of data comes in the form of loyalty programs
such as a grocery store club card, social sentiments like those found on Yelp, social media, and
even weather patterns. These data were arbitrary before and difficult to impossible to analyze.
New technologies however, are enabling us to “slice and dice” these data to find patterns,
trend, seasonality, and more. All of these form the new understanding of what a demand signal
“may” actually be.
With an understanding of demand signals, the user can now explore the emerging
technology found in the Demand Signal Repository. The DSR is a data warehouse that is
designed to cleanse and integrate demand data. This enables the end user to quickly and
effectively access the data and begin exploration in a meaningful way. The significance of this
technology is that this storage procedure is completely automated. Regardless of the file
format that the data arrives in, the DSR converts it into a common form.
DSR’s incorporate a form of data mining, which is the automatic discovery patterns
found within a data source. It involves predicting likely outcomes, the creation of information
that can be acted upon, and its focus is on large data sets and databases. It has also been called
Knowledge Discovery in Data or KDD. This technique is designed to answer questions that
cannot be answered with a simple query or simple reporting techniques. The automatic
discovery implements a model that leverages algorithms in its discovery. These models are
typically built by IT and analytical professionals. Another form is grouping. This method
attempts to identify natural groupings within the data.
DSR’s also provide a platform for On-Line Analytical Processing or OLAP. OLAP is a
complementary activity to data mining. This methodology is used for data summarization, cost
allocation, and statistical analyses. The purpose of this type of system is to provide a
multidimensional view of data to provide structure. The data is stored in a multidimensional
database that is often referred to as an OLAP cube. Two of the leading products offering this
technology are Oracles’s “Express Server” and Hyperion Solution’s “Essbase.” Another thing
that must be understood about OLAP is that it is far more granular than Data mining, and it
focusses on understanding what the data are telling us. As we will see in the following
paragraphs, many new technologies are being combined to provide businesses with a new
understanding of their operating environments.
The overwhelming amount of data that is being generated in today’s world also requires
the use of another emerging technology called the cloud. DSR’s exist in a cloud based format.
Cloud computing as it is called, utilizes a network of remote servers which are hosted on the
internet. Data can be stored, managed, and processed on these instead of on the company’s
computers. This technology eliminates the burden of company’s needing to build their own
massive data repository. This means that users will not need to work off of their own hard
drives, and instead have access to dedicated, internet based storage and processing power. This
Software-as-a-Service or SaaS, is typically subscription based. Business users are likely to also
need to subscribe to Infrastructure-as-a-Service or IaaS. IaaS allows IT to operate a data center
in the cloud, and all hardware management concerns are performed by the cloud provider. The
thing to take home from this technology for most users, is that it is far cheaper than developing
your own infrastructure.
What this means to a forecaster or IT professional is that he can produce forecasts and
what if analyses far more rapidly and from far more sources nearly in real-time. Current
systems that are leveraging this technology include data from point-of-sale, wholesalers, supply
chains, and customer loyalty. Marketing departments are using them to analyze the
effectiveness of promotions. Financial professionals are using the data to interpret financial
results and regional performance. Perhaps most importantly, top executives are gaining a more
complete top-down view of the organization. With all of this data in a common format in near
real-time, managers are able to make better decisions with significantly reduced lag.
Forecasting professionals were among the first to begin adopting these technologies and with
good reason. The average forecasting error by industry is as follows: Retail 13%, electronics
29%, aerospace 28%, manufacturing 39%. Reducing these errors is paramount so that
businesses can function more efficiently. Readily available access to as much “clean” data as
possible is changing the way these people do their jobs.
Reporting/DSV
BI reporting is a significant advancement over the simple charts that Excel users once
employed. The tools available to IT and analytics professionals are far more robust in terms of
visualization and capability. Furthermore, the DSR’s discussed in the previous paragraphs are
being empowered by yet another innovation: Demand Signal Visualizations.
The Demand Signal Visualization or DSV, brings the visual aspect of reporting to the DSR.
These systems allow the user to transform the data contained in the DSR into a visual
representation. This allows for exploration, analysis and insight. A lot can be gained from these
systems, especially since many senior executives are not IT or analytical professionals. The
visual reporting format allows users to build charts and graphs with data from all departments
in an organization. This type of visualization can provide insight in areas that a company should
be focusing, where it can improve, and if action needs to be taken. DSV’s truly empower the
top-down view and allow an executive or a forecaster to compare departments performance,
human resources, safety concerns, or almost anything else that you can imagine. Furthermore,
many of these visualizations can be delivered to your boss’s mobile phone! Imagine getting an
email or text from your boss who is in a conference out of town and he needs an understanding
of specific parts of the organization. You send it directly to his phone before the conference is
over! This technology is going to make IT and analytics professionals heroes in the workplace.
Discovery/DSA
The third and final layer of these technologies is very new. It is called Demand Signal
Analytics or DSA’s. The DSA is capable of pulling data from the DSR and DSA’s to produce visual
and predictive analytics. The claim for DSA’s, is that the end user does not need common
forecasting techniques such as sampling and subsetting to leverage their power. Sampling is a
method for selecting a “subset of individuals from within a statistical population to estimate
characteristics of the entire population.” Subsetting is the retrieval of the relevant parts of large
files that are of particular interest or will serve a specific purpose. The DSA’s are designed to
automatically build the most appropriate model for whatever data needs to be examined. This
is only half true however, because someone in IT or analytics will need to specify how the data
should be “sliced and diced” and what type of analytics are the most appropriate for each type
of model. This fact provides job security now and for the future to those working in these fields.
DSA’s use a combination of descriptive, predictive, and prescriptive analytics.
Descriptive analytics attempt to quantitatively describe the important features of data.
Predictive analytics utilizes a wide variety of statistical techniques such as predictive modeling,
machine learning, and data mining. This technique is used to exploit patterns in a company’s
historical data to identify risks and opportunities. Generally, they focus on the relationships
between variables and are used in forecasting a variety of uncertainties. Prescriptive analytics
use a combination of the previous three to suggest options for management. They are used to
answer to determine why things will happen and how management may be able to take
advantage of an opportunity or avoid a risk. The combination of these can be used to uncover
actionable insights by determining what the data are telling us and how it can be leveraged.
These technologies are a driving force behind the data revolution and self-service analytics
because they are becoming increasingly easy to access and implement. In the near future
businesses will be far more agile than what they are today as a result. Those who do not adopt
these technologies will find it immensely difficult to compete.
Much of this power is being utilized for demand sensing. This concept goes beyond just
trend and seasonality however, and answers many important questions such as: How much did
the consumer pay? How much would they be willing to pay? What advertisements did the
consumer see? How was the product promoted in a given store? What is the economy like in a
given market?
These questions as well as others allow business professionals to “sense” the level of
demand, and how it might change. The sources of downstream and upstream data are
providing this type of insight in ways never before thought possible. Additionally, they are
enabling a powerful form of a concept known as demand shaping.
Demand shaping the measuring of relationships of consumer’s demand with sales. The
goal of these is to measure the effectiveness of promotions and marketing events so that they
can be used to shape future demand. Some of the data being measured are promotions, sales
tactics, marketing events, changes in the product mix, introductions of new products, and
more.
The combination of demand sensing and demand shaping, when implemented using
DSR’s, DSV’s, and DSA’s, is reduced latency. In other words, a reduction of a business’s reaction
time. These improved capabilities will be used in harmony to model and forecast far more
frequently than the typical rate seen today.
The business that effectively implements these technologies will realize a vast number
of benefits. A few examples of those benefits are:

Improved demand forecast accuracy.

Enhanced Demand sensing and shaping activities.

Reduced Out-of-Stocks.

Faster discovery of product category demand changes.

Improved evaluation of new product information from the integration of sentiment
analysis.

Increased promotion effectiveness.

Lower inventory and safety stock levels.
The importance of the adoption of these technologies is constantly increasing. An
inaccurate forecast could mean that logistics doesn’t ship the appropriate quantity of product
which results in insufficient inventory levels. Sales may be lost in the event of a shortage,
resulting in reduced revenues. Carrying cost is increased in the event of excess stock which
increases cost and cuts into a company’s margins.
As we have seen the data revolution is already underway. The major problem that
businesses are facing now and in the near future lies in the adoption and implementation of
these technologies. Many businesses describe their adaptation as “in its infancy.” Current and
future professionals will face this integration and implementation and should be preparing for it
now.
A hurdle that many companies have encountered is that large portions of their historical
data are useless. That may be because it was improperly entered, stored, or changed. Excluding
the “best-in-class” businesses, the determination of what data are usable and what data are
useless are the questions that the majority of businesses are trying to answer. The next tier has
begun to implement DSR’s but has not yet incorporated DSV’s and DSA’s, at least not in a
significant way. These facts mean that only the top echelon of companies have completed this
migration. All of that is set to change with the combination of cloud technology, self-service
analytics, data mining, OLAP’s, DSR’s, DSV’s and DSA’s. A thorough understanding of this
combination of technologies provides us with a realistic view of what the future will hold. The
time to learn these technologies is now, right now. IT and analytical professionals are growing
in demand. This paper provides a good insight as to why, how, and the potential impact on the
fundamental ways that business operations will change. The bottom line: Businesses and
professionals need to start preparing now!
Citations
(To be properly formatted in final paper)
Innovations in Business Forecasting: Predictive Analytics – Journal of BusinessForecasting –
Charles W. Chase Jr. – Summer 2014, Vol. 33, pp. 26-30, 32
Visualization – Key to Predictive Analytics – Journal of Business Forecasting – Dr. Larry Lapide –
Winter 2014/2015, Vol 33, pp 34 - 38
“The Black Swan – The Impact of the Highly Improbable” – Nassim Taleb
Power Pivot and Power BI – Rob Collie & Avichal Singh
http://www.pcmag.com/article2/0,2817,2496292,00.asp
http://www.pcmag.com/article2/0,2817,2372163,00.asp
https://en.wikipedia.org/wiki/Subsetting
http://docs.oracle.com/cd/B28359_01/datamine.111/b28129/process.htm#CHDFJEJI
https://en.wikipedia.org/wiki/Prescriptive_analytics
https://en.wikipedia.org/wiki/Predictive_analytics
https://en.wikipedia.org/wiki/Descriptive_statistics
http://www.businesscomputingworld.co.uk/the-history-of-business-intelligence-infographic/
http://searchdatamanagement.techtarget.com/definition/business-intelligence
http://searchdatamanagement.techtarget.com/definition/OLAP