Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
What is Business Intelligence? Business Intelligence or BI, is a rapidly advancing technology-driven process used for data analysis. The tools and skills involved compile data to deliver actionable information to managers and executives. Some examples of these tools are SSAS Tabular, Power BI Desktop, and Power Pivot. SSAS Tabular and Power Pivot are similar in that they compile data and allow users to write calculations (measures), on vast amounts of information. Desktop applications compile data in a similar way except with emphasis on visualizations. The benefits realized from these technologies are faster decision making, optimized internal business processes, and increased efficiency. These benefits allow companies to realize competitive advantages. Firms that are adopting these tools are realizing these advantages quickly. Even these technologies are only the bare bones of what the future holds. BI technology is progressing at an astounding rate and if it hasn’t yet, it will impact the way you do business. Though we have only scraped the surface, the reader will soon realize that these new BI technologies are the beginning of the data revolution! Companies that learn to use their data effectively will win in the market, those who do not will fail. The term Business Intelligence first appeared in the 1860’s. It was not used in the current context however, until 1989 when Howard Dresner proposed the term as an umbrella category. He defined it as a set of data analysis techniques that support business decisionmaking processes. BI technologies evolved from executive information systems and decision support systems. Others have defined it as a set of “computer-based techniques used in identifying, extracting, and analyzing business data – such as sales revenue by products and/or departments, or by associated costs and incomes.” Most recently its use has been extended to business users and IT decision makers equally. Its purpose is to provide historical, current and predictive views of a business. Business Intelligence is most commonly used for data mining, analytics, reporting, process mining, benchmarking, performance management and more. The current understanding of this technology began in the early 1990s with enterprise resource planning, which aimed to link corporate databases with client/server technologies. The goal was to support the selling process, finance, human resources, and other aspects of business. The rise of the internet allowed these technologies to reach heights never before imagined. It wasn’t until 1997 that the term “Business Intelligence” became widely used. That year forecasters began to widely adopt these ideas and leverage them to perform predictive analyses. The need for these technologies was underscored in 1994 when news about Wal-Mart reported that its headquarters housed 460 terabytes of data, or more than double the amount of the internet at that time! By 2010 67% of “best in class” companies boasted some form of self-service BI. Current forecasts show that the use of business intelligence will continue to grow and evolve. This means that it is becoming increasingly important for business users to learn, embrace, and implement these technologies into their day-to-day activities. The power of these technologies has evolved to a point where they are accessible to even the smallest of businesses. Business Intelligence was once reserved for large companies with vast amounts of capital and technological capability. Today a single user can access the same capabilities at their desktop with services from Hadoop, Oracle, and Amazon. Self-service analytics are being used in ways that were not possible ten years ago. This vast availability and ease of access is what is defining the data revolution. Data storage/DSR BI has changed significantly in the past 10 years. Analysts and IT professionals are gaining access to something called a Demand Signal Repositories. To understand what these are, the user must first understand what a demand signal is. Some traditional demand signals are consumption and inventory shipments and stock levels. Consumption is a type of downstream data source and describes the amount being purchased at a given time. Downstream is a term used to describe the data that we receive from the end of the value chain. Conversely, upstream data is the data that we are receiving from our suppliers, manufacturing plants, and possibly warehouses; depending on where we are viewing the value stream from. Inventory and shipment stock levels are the amounts of product that are being shipped or are currently available in store or in the warehouse. The evolution of technology is providing us access and understanding of another type of demand signal, the unstructured form. This type of data comes in the form of loyalty programs such as a grocery store club card, social sentiments like those found on Yelp, social media, and even weather patterns. These data were arbitrary before and difficult to impossible to analyze. New technologies however, are enabling us to “slice and dice” these data to find patterns, trend, seasonality, and more. All of these form the new understanding of what a demand signal “may” actually be. With an understanding of demand signals, the user can now explore the emerging technology found in the Demand Signal Repository. The DSR is a data warehouse that is designed to cleanse and integrate demand data. This enables the end user to quickly and effectively access the data and begin exploration in a meaningful way. The significance of this technology is that this storage procedure is completely automated. Regardless of the file format that the data arrives in, the DSR converts it into a common form. DSR’s incorporate a form of data mining, which is the automatic discovery patterns found within a data source. It involves predicting likely outcomes, the creation of information that can be acted upon, and its focus is on large data sets and databases. It has also been called Knowledge Discovery in Data or KDD. This technique is designed to answer questions that cannot be answered with a simple query or simple reporting techniques. The automatic discovery implements a model that leverages algorithms in its discovery. These models are typically built by IT and analytical professionals. Another form is grouping. This method attempts to identify natural groupings within the data. DSR’s also provide a platform for On-Line Analytical Processing or OLAP. OLAP is a complementary activity to data mining. This methodology is used for data summarization, cost allocation, and statistical analyses. The purpose of this type of system is to provide a multidimensional view of data to provide structure. The data is stored in a multidimensional database that is often referred to as an OLAP cube. Two of the leading products offering this technology are Oracles’s “Express Server” and Hyperion Solution’s “Essbase.” Another thing that must be understood about OLAP is that it is far more granular than Data mining, and it focusses on understanding what the data are telling us. As we will see in the following paragraphs, many new technologies are being combined to provide businesses with a new understanding of their operating environments. The overwhelming amount of data that is being generated in today’s world also requires the use of another emerging technology called the cloud. DSR’s exist in a cloud based format. Cloud computing as it is called, utilizes a network of remote servers which are hosted on the internet. Data can be stored, managed, and processed on these instead of on the company’s computers. This technology eliminates the burden of company’s needing to build their own massive data repository. This means that users will not need to work off of their own hard drives, and instead have access to dedicated, internet based storage and processing power. This Software-as-a-Service or SaaS, is typically subscription based. Business users are likely to also need to subscribe to Infrastructure-as-a-Service or IaaS. IaaS allows IT to operate a data center in the cloud, and all hardware management concerns are performed by the cloud provider. The thing to take home from this technology for most users, is that it is far cheaper than developing your own infrastructure. What this means to a forecaster or IT professional is that he can produce forecasts and what if analyses far more rapidly and from far more sources nearly in real-time. Current systems that are leveraging this technology include data from point-of-sale, wholesalers, supply chains, and customer loyalty. Marketing departments are using them to analyze the effectiveness of promotions. Financial professionals are using the data to interpret financial results and regional performance. Perhaps most importantly, top executives are gaining a more complete top-down view of the organization. With all of this data in a common format in near real-time, managers are able to make better decisions with significantly reduced lag. Forecasting professionals were among the first to begin adopting these technologies and with good reason. The average forecasting error by industry is as follows: Retail 13%, electronics 29%, aerospace 28%, manufacturing 39%. Reducing these errors is paramount so that businesses can function more efficiently. Readily available access to as much “clean” data as possible is changing the way these people do their jobs. Reporting/DSV BI reporting is a significant advancement over the simple charts that Excel users once employed. The tools available to IT and analytics professionals are far more robust in terms of visualization and capability. Furthermore, the DSR’s discussed in the previous paragraphs are being empowered by yet another innovation: Demand Signal Visualizations. The Demand Signal Visualization or DSV, brings the visual aspect of reporting to the DSR. These systems allow the user to transform the data contained in the DSR into a visual representation. This allows for exploration, analysis and insight. A lot can be gained from these systems, especially since many senior executives are not IT or analytical professionals. The visual reporting format allows users to build charts and graphs with data from all departments in an organization. This type of visualization can provide insight in areas that a company should be focusing, where it can improve, and if action needs to be taken. DSV’s truly empower the top-down view and allow an executive or a forecaster to compare departments performance, human resources, safety concerns, or almost anything else that you can imagine. Furthermore, many of these visualizations can be delivered to your boss’s mobile phone! Imagine getting an email or text from your boss who is in a conference out of town and he needs an understanding of specific parts of the organization. You send it directly to his phone before the conference is over! This technology is going to make IT and analytics professionals heroes in the workplace. Discovery/DSA The third and final layer of these technologies is very new. It is called Demand Signal Analytics or DSA’s. The DSA is capable of pulling data from the DSR and DSA’s to produce visual and predictive analytics. The claim for DSA’s, is that the end user does not need common forecasting techniques such as sampling and subsetting to leverage their power. Sampling is a method for selecting a “subset of individuals from within a statistical population to estimate characteristics of the entire population.” Subsetting is the retrieval of the relevant parts of large files that are of particular interest or will serve a specific purpose. The DSA’s are designed to automatically build the most appropriate model for whatever data needs to be examined. This is only half true however, because someone in IT or analytics will need to specify how the data should be “sliced and diced” and what type of analytics are the most appropriate for each type of model. This fact provides job security now and for the future to those working in these fields. DSA’s use a combination of descriptive, predictive, and prescriptive analytics. Descriptive analytics attempt to quantitatively describe the important features of data. Predictive analytics utilizes a wide variety of statistical techniques such as predictive modeling, machine learning, and data mining. This technique is used to exploit patterns in a company’s historical data to identify risks and opportunities. Generally, they focus on the relationships between variables and are used in forecasting a variety of uncertainties. Prescriptive analytics use a combination of the previous three to suggest options for management. They are used to answer to determine why things will happen and how management may be able to take advantage of an opportunity or avoid a risk. The combination of these can be used to uncover actionable insights by determining what the data are telling us and how it can be leveraged. These technologies are a driving force behind the data revolution and self-service analytics because they are becoming increasingly easy to access and implement. In the near future businesses will be far more agile than what they are today as a result. Those who do not adopt these technologies will find it immensely difficult to compete. Much of this power is being utilized for demand sensing. This concept goes beyond just trend and seasonality however, and answers many important questions such as: How much did the consumer pay? How much would they be willing to pay? What advertisements did the consumer see? How was the product promoted in a given store? What is the economy like in a given market? These questions as well as others allow business professionals to “sense” the level of demand, and how it might change. The sources of downstream and upstream data are providing this type of insight in ways never before thought possible. Additionally, they are enabling a powerful form of a concept known as demand shaping. Demand shaping the measuring of relationships of consumer’s demand with sales. The goal of these is to measure the effectiveness of promotions and marketing events so that they can be used to shape future demand. Some of the data being measured are promotions, sales tactics, marketing events, changes in the product mix, introductions of new products, and more. The combination of demand sensing and demand shaping, when implemented using DSR’s, DSV’s, and DSA’s, is reduced latency. In other words, a reduction of a business’s reaction time. These improved capabilities will be used in harmony to model and forecast far more frequently than the typical rate seen today. The business that effectively implements these technologies will realize a vast number of benefits. A few examples of those benefits are: Improved demand forecast accuracy. Enhanced Demand sensing and shaping activities. Reduced Out-of-Stocks. Faster discovery of product category demand changes. Improved evaluation of new product information from the integration of sentiment analysis. Increased promotion effectiveness. Lower inventory and safety stock levels. The importance of the adoption of these technologies is constantly increasing. An inaccurate forecast could mean that logistics doesn’t ship the appropriate quantity of product which results in insufficient inventory levels. Sales may be lost in the event of a shortage, resulting in reduced revenues. Carrying cost is increased in the event of excess stock which increases cost and cuts into a company’s margins. As we have seen the data revolution is already underway. The major problem that businesses are facing now and in the near future lies in the adoption and implementation of these technologies. Many businesses describe their adaptation as “in its infancy.” Current and future professionals will face this integration and implementation and should be preparing for it now. A hurdle that many companies have encountered is that large portions of their historical data are useless. That may be because it was improperly entered, stored, or changed. Excluding the “best-in-class” businesses, the determination of what data are usable and what data are useless are the questions that the majority of businesses are trying to answer. The next tier has begun to implement DSR’s but has not yet incorporated DSV’s and DSA’s, at least not in a significant way. These facts mean that only the top echelon of companies have completed this migration. All of that is set to change with the combination of cloud technology, self-service analytics, data mining, OLAP’s, DSR’s, DSV’s and DSA’s. A thorough understanding of this combination of technologies provides us with a realistic view of what the future will hold. The time to learn these technologies is now, right now. IT and analytical professionals are growing in demand. This paper provides a good insight as to why, how, and the potential impact on the fundamental ways that business operations will change. The bottom line: Businesses and professionals need to start preparing now! Citations (To be properly formatted in final paper) Innovations in Business Forecasting: Predictive Analytics – Journal of BusinessForecasting – Charles W. Chase Jr. – Summer 2014, Vol. 33, pp. 26-30, 32 Visualization – Key to Predictive Analytics – Journal of Business Forecasting – Dr. Larry Lapide – Winter 2014/2015, Vol 33, pp 34 - 38 “The Black Swan – The Impact of the Highly Improbable” – Nassim Taleb Power Pivot and Power BI – Rob Collie & Avichal Singh http://www.pcmag.com/article2/0,2817,2496292,00.asp http://www.pcmag.com/article2/0,2817,2372163,00.asp https://en.wikipedia.org/wiki/Subsetting http://docs.oracle.com/cd/B28359_01/datamine.111/b28129/process.htm#CHDFJEJI https://en.wikipedia.org/wiki/Prescriptive_analytics https://en.wikipedia.org/wiki/Predictive_analytics https://en.wikipedia.org/wiki/Descriptive_statistics http://www.businesscomputingworld.co.uk/the-history-of-business-intelligence-infographic/ http://searchdatamanagement.techtarget.com/definition/business-intelligence http://searchdatamanagement.techtarget.com/definition/OLAP