Download Variables Storage, Type, Declaration.

Foundational Software
Systems
A Brief Overview
Copyright © 1998-2016 Curt Hill
Introduction
• Certain classes of software need to
be understood by all technical
people
• Operating Systems
• Servers
• Databases and Data Warehouses
• Enterprise Resource Planning
• Customer Relationship Management
Copyright © 1998-2016 Curt Hill
Outroduction
•
•
•
•
•
•
What this presentation is not about:
Virtualization
Cloud computing
Service Oriented Architecture
Software as a Service
These are important topics to be
considered at another time
Copyright © 1998-2016 Curt Hill
OS
• Foundation of the foundation
• Every general purpose computer
has one
– Many special purpose computers as
well
• Applications are why we buy the
computer, but the operating system
is required to run the applications
Copyright © 1998-2016 Curt Hill
What does an OS do?
•
•
•
•
•
Execute application programs
Provide services to these programs
Establish and manage a file system
Utility functions
Many functions that are used by
most programs will eventually be
incorporated into the OS
Copyright © 1998-2016 Curt Hill
Execute application
programs
• Interpret commands
– Interpret typed commands or mouse
actions
• Load and execute programs
• Provide services to the running
programs
– The exact description of these services
distinguishes different OSs
Copyright © 1998-2016 Curt Hill
Provide services to the
program
• Memory management
– Memory allocation
– Virtual or fixed memory
• I/O management
– Manage file activity
– Often the most visible and important
• Task Scheduler
– Inter-process communication
– Managing priorities
Copyright © 1998-2016 Curt Hill
Establish and manage a file
system
• Determine how things are stored on
disk
– Do we have a directory or just search?
– Are things stored in a contiguous
fashion or scattered?
– How are things named?
– How are things accessed?
• Interrupt processing
• Utility functions
– Prepare a disk for use
– Copy/move/delete files
– Show contents of a file or disk
Copyright © 1998-2016 Curt Hill
How are files accessed?
• Access to files is a program service
– Program says that it wants access to a
file of some filename
– System finds it
– Gets it ready
– Gives it to the program one chunk at a
time
• Several things should be
transparent to program
• Buffering
• Devices Copyright © 1998-2016 Curt Hill
Utility functions
• Protection
– Files from deletion by someone other
than owner
– Memory of one program from another
• Hardware manipulation
– Mount disks and tapes
• Miscellaneous
– Error detection
– Multitasking
– Accounting
Copyright © 1998-2016 Curt Hill
Pieces of the operating
system
•
•
•
•
•
•
•
CPU management
Memory management
Process management
I/O management
User management
File management
Others
Copyright © 1998-2016 Curt Hill
OS Layers
• Most OSs are onion shaped
– For historical and maintenance reasons
– Historically the innermost was all that
was available
• Each layer then communicates with
adjacent layers through well defined
paths
– Well defined in terms of this particular
OS
• Layers: Kernel, Service, Command
Copyright © 1998-2016 Curt Hill
Kernel is the innermost layer
• Low level activities only
–
–
–
–
Load and/or execute a program
Device allocation and access
Process management
Low level file access
• This is the lowest level interface to the hardware
and the only one that touches the hardware
directly
• In DOS there is the BIOS which amounts to a
kernel
• The kernel is usually always resident in the
memory
• Basic process management is usually here,
since we need to make each program we load a
process
Copyright © 1998-2016 Curt Hill
Service layer
• Set of routines that provide services
– Use the kernel
– This is where the traditional file manipulation
commands are executed, such as opening
and closing files, reading and writing records
– Entire file copying is usually done at the next
higher level
• The service layer represents the services
that are available to a program
• The service layer is also designed to
generalize the interface between devices
– A CRT, disk and tape are wildly different, yet
we would like their interface to look similar
Copyright © 1998-2016 Curt Hill
Service Layer
• In the assembly language interface
each of the service calls becomes a
macro or subroutine call
• In a HLL it is usually built into the
command part of the language or is
in a library of some sort
• Some process management is here
as well
– Inter-process communication
Copyright © 1998-2016 Curt Hill
Command layer
• The interface that allows the user to
interface with the system directly
• This includes command interpretation
• This is Command.com in DOS or program
manager in Windows
• Many utility functions are placed in this
layer
– The DOS Format is a command in this layer
• It performs an essential function, but is its
own program
Copyright © 1998-2016 Curt Hill
Familiar OSs
• Windows
– Several versions from personal to
server
• UNIX
– Bell Labs and BSD
• Mac OS
– OS 10 is a BSD UNIX with specialized
Mac GUI on top
• LINUX
– UNIX like alternative to UNIX
Copyright © 1998-2016 Curt Hill
LINUX Again
• LINUX is open source so there are
many distributions of it:
–
–
–
–
–
Debian
Ubuntu
Fedora
SUSE
Red Hat Enterprise
• There are very many others
Copyright © 1998-2016 Curt Hill
Servers
• Client server computing is the basis
of the Internet
– Among other things
• Requires two computers
– A client
– A server
• Communication medium
• The illusion is that each client
computer thinks it is has exclusive
access to the server
Copyright © 1998-2016 Curt Hill
The Paradigm
•
•
•
•
•
Server provides one service
Waits for requests for that service
Client initiates a request
Server replies to the request
Information may flow in both
directions
• Many clients may make requests
with just the one server
Copyright © 1998-2016 Curt Hill
Why Client - Server?
• Specialization
– The client handles the user interface
using local computational ability
– The server manages the data and
communication
• Sharing
– One server may support many clients
– Services provided are effectively
shared
– Data may be maintained in only one
location Copyright © 1998-2016 Curt Hill
Clients
• An application program that runs on
the local computer
• Able to use the local computer for
doing work
• When invoked by a user, it initiates
communication with a server
• Handles all the user interface issues
• Only used for a single session
Copyright © 1998-2016 Curt Hill
Example
• The ubiquitous client is the web
browser
• Several functions:
• Communicate with server
• Display HTML/XML
• Interpret client-side languages,
such as JavaScript
• The trend is for the web browser to
be the universal interface to every
kind of server on the web
Copyright © 1998-2016 Curt Hill
Servers
• Specialized program that typically
provides only one service
• May handle multiple client requests
simultaneously (or pseudo
simultaneously)
• Interacts with users in many sessions
– Both simultaneously and over long time
periods
• Waits for a request and then produces the
reply
Copyright © 1998-2016 Curt Hill
Example
• The web server
• Waits for requests using Hyper Text
Transmission Protocol
• Produces results in HTML/XML
• May handle many clients
simultaneously
Copyright © 1998-2016 Curt Hill
Connection Diagrammed
Server
Request
Request
Client
Reply Reply Request
Client
Copyright © 1998-2016 Curt Hill
Reply
Client
Interaction Diagrammed
Time
Client
Processing
Request
Wait
Reply
Server
Wait
Wait
• The server is not waiting if it is
processing other requests
• The client may also perform other
processing while waiting for the
reply
Copyright © 1998-2016 Curt Hill
Multiple tiers
• The classic client-server is a two tier
system
• The client is one tier and the server
another
• Three or more tiers are also
possible
• The middle server functions as a
– Server to the client
– Client to other servers
– Known as middleware
• Often used to distribute load
Copyright © 1998-2016 Curt Hill
A Three Tier Approach
Server
Server
Server
Server
Application Server
Client
Client
Client
Copyright © 1998-2016 Curt Hill
Client
Examples of Multi-Tier
• Authentication server
– Maintains a connection to verify user
– Funnels all requests to appropriate
server
• Shopping cart
– Records purchases
– The product lookup/purchase handled
by other servers
• Distributed Database Management
System
Copyright © 1998-2016 Curt Hill
Horizontal Distribution
• Multi-Tier is a vertical distribution
• A Horizontal distribution may also be
effective
• A single server owns the web
address: www.x.com
• It then distributes all further
requests to a rotation of identical
web servers
–
–
–
–
www1.x.com
www2.x.com
…
wwwN.x.com
Copyright © 1998-2016 Curt Hill
Horizontal Distribution
www.x
First
Request
www1.x www2.x
… wwwN.x
Subsequent
Request
Client Client Client Client Client Client
Copyright © 1998-2016 Curt Hill
Database
• The web server may be the most
common server type, but the
database server is the workhorse
• It is the foundation for many other
important classes of programs
Copyright © 1998-2016 Curt Hill
Historical Perspective
• Actions we might want on any file of fixed
records:
–
–
–
–
Create
Update (add, remove, change records)
Sort
Generate any of several reports
• Each of these would be a program for an
overworked programming staff
– Typically a COBOL program
Copyright © 1998-2016 Curt Hill
Database
• The key
– All these programs are about the same
– All that changes is the underlying file
• The solution
– Describe the file in a general way
• This is the metadata – data that describes
data
– Generate a program that handles the
file based on the description
Copyright © 1998-2016 Curt Hill
How to describe a file
• A file is a collection of records
• Each record is a collection of fields
– Typically only one type of record in a
file
• Each field is described by a:
– Name
– Type
• For example numeric, string, boolean etc.
– Length
• Booleans have a predefined length, others
require specification
Copyright © 1998-2016 Curt Hill
Database Advantages 1
• Data independence
– Application program no longer need
some or all of the files
– Do not know or care how data is stored,
aka abstraction
– Simplifies application development
• Efficient access
– The DBMS employs sophisticated
access techniques seldom used by
normal programmers
Copyright © 1998-2016 Curt Hill
Database Advantages 2
• Integrity constraints
– The DBMS may check data in a way
seldom done in normal file processing
– Eg. Account validity
• Security
– A DBMS may enforce requirements on
who can access the data and in what
way
Copyright © 1998-2016 Curt Hill
Database Advantages 3
• Administration
– Minimize redundancy
– Manage sharing of the data
– Optimize for the enterprise, not a small
group
– Easier to backup the data
• Concurrent access
– Manages the simultaneous update
problem
Copyright © 1998-2016 Curt Hill
Functions of a Database
•
•
•
•
•
•
•
•
•
Data storage and retrieval
User-accessible catalog
Transaction support
Concurrency support
Recovery services
Authorization services
Support for data communication
Integrity services
Data independence services
Copyright © 1998-2016 Curt Hill
Data storage and retrieval
• While hiding the file structure, users
may:
– Add
– Delete
– Update in place
• Most relational databases support a
common language
– Structured Query Language
Copyright © 1998-2016 Curt Hill
SQL
• A language where a question is
asked of the database and it
provides an answer:
– Pronounced Sequel or spelled out
– Usually state what is wanted rather
than how to get it
• The Data Manipulation Language
usually includes both query and
update facilities
Copyright © 1998-2016 Curt Hill
Transaction Support
• The unit of update is a transaction
– All or nothing
– No partial updates are allowed
• The DBMS must guarantee the
integrity of a transaction, even in the
face of:
– Program errors
– Incorrect input
– System crashes
Copyright © 1998-2016 Curt Hill
Transactions: ACID
• Atomic
– Transaction perceived to be indivisible
• Consistent
– Transforms database from one consistent
state to another
• Isolated
– Understandable without regards to any
other agents
• Durable
– Once committed permanence is
guaranteed even with system crashes
Copyright © 1998-2016 Curt Hill
Types of database
• A large organization often have two
distinct types of databases
• A transactional or operational
database
• An analytical database or data
warehouse
• NoSQL
• These have separate uses
Copyright © 1998-2016 Curt Hill
Database uses
• The database is about transactions
– Maintaining current knowledge
• The data warehouse is about
decision support
– Spotting trends
• NoSQL is much harder to classify,
any of these
–
–
–
–
Massive data
Unique organizations
Distributed
Among many others
Copyright © 1998-2016 Curt Hill
Characteristics
• Operational database have:
–
–
–
–
Strict performance requirements
Predictable workloads
Small units of work
High utilization
• Data warehouses are contrary in
every respect
Copyright © 1998-2016 Curt Hill
Warehouses
• A data warehouse
– Subject oriented, integrated, nonvolatile, time-variant collection of data
in support of management decsions
• Data warehouses support
– OLAP (OnLine Analytical Processing)
– DSS/EIS (Decision Support Systems or
Executive Information Systems)
– Data mining
Copyright © 1998-2016 Curt Hill
What is Data Mining?
• Exploratory data analysis based on a data
warehouse
– Knowledge Discovery in Databases (KDD)
• Data Mining extracts previously unknown
and potentially useful information
– Rules, constraints, correlations, patterns,
signatures and irregularities
• The goal is to automate the methods for
finding these in the data
Copyright © 1998-2016 Curt Hill
Goals of Data Mining
• Prediction of future behaviors
– Seasonal or non-seasonal trends
– How will consumers respond to
discounts?
– Allows the enterprise to be ready
• Identification of item, event or
activity
– Intruders may be identified by the files
they access or programs they use
Copyright © 1998-2016 Curt Hill
Goals Again
• Classification of categories of users
or products
– Shoppers may be categorized as:
•
•
•
•
Discount seeking
Rush
Regular
Attached to certain brand names
– The store may be made more friendly to
such
• Optimize the use of time, space,
materials and
money
Copyright © 1998-2016 Curt Hill
Knowledge Discovery
• There are several types of
discoverable knowledge:
• Association Rules
• Classification hierarchies
• Sequential patterns
• Time series patterns
• Clustering
Copyright © 1998-2016 Curt Hill
Databases
• The two types of databases are the
transactional and analytical
– The day to day transactional feeds the
analytical
• We do not want knowledge
discovery, data mining and the like to
occur on the transactional database
– Could seriously impede performance
• Either might be used in the next two
applications
– Transactional is more likely
Copyright © 1998-2016 Curt Hill
• Data
Some Definitions
– Small measurable pieces of information
– Foundation of knowledge
• Information
– Organized data
– Putting data into meaningful patterns
• Knowledge
– A higher aggregation of information
and data that enables interpretation
– Ability to use information
– Sets of rules and relationships
Copyright © 1998-2016 Curt Hill
Example
• Any numbers or text is data
– Such as: 324 3.83
• It becomes information when we
know its context
– Such as these are the number of items
in stock and how much we paid per unit
• Knowledge is when we take this
information and combine it with
processes and rules
– Computing our price for selling
something made out of these
Copyright © 1998-2016 Curt Hill
Data Mart
• A data warehouse restricted to a
single subject
– Such as a group or line of business
within a larger organization
• A dependent data mart obtains its
data from the central organizational
data warehouse
• An independent data mart obtains
data from other sources
– No overlap with the data warehouse
Copyright © 1998-2016 Curt Hill
Knowledge Warehouse
• An architectural integration of
knowledge management, decision
support, artificial intelligence and
data warehousing
• Built on top of a data warehouse
• Must also encode the rules and
processes of the business
– These rules used to only be in the
minds of employees
– Next captured on documents
Copyright © 1998-2016 Curt Hill
ERP
• Enterprise Resource Planning
• The key idea behind database
technology is that the process of
manipulating files is basically the
same
– The only thing that changes is the file
format
• The key to ERP is that many
business processes are very similar
– Payroll is just payroll
• Took off because of the Euro and
Y2K threatCopyright © 1998-2016 Curt Hill
ERP Areas
•
•
•
•
•
•
•
•
Financial Accounting
Management Accounting
Human Resources
Manufacturing
Supply Chain Management
Project Management
Customer Relationship Management
These were all separate programs at
one time
Copyright © 1998-2016 Curt Hill
Configuration
• Most systems need to be customized
by programmers modifying the code
• Most ERPs may be configured to
customize to a particular situation
• This configuration is usually much
easier than code modification
• The program typically handles best
practices, but the configuration can
take into account the idiosyncrasies
of this business
Copyright © 1998-2016 Curt Hill
Simple Example
• The form of a product number
drastically varies between
companies
– Usually groups of letters and digits
• Configuring the system to store and
validate the type used by a
particular company needs to be a
simple configuration
Copyright © 1998-2016 Curt Hill
CRM
• Customer Relationship Management
• The notion of a call center
• A past, present or future customer
calls
• A representative, with no previous
knowledge of this customer, has to
deal with them
• CRM puts all the information
necessary to help in front of the
representative, regardless of the
purpose of the call
Copyright © 1998-2016 Curt Hill
Types of calls
• Sales
– Availability, delivery, costs and
discounts
• Support after the sale
– Warranty, replacements, returns
• Information and marketing
– Request for information
• Appointments for sales force
• Analysis of customer or products
Copyright © 1998-2016 Curt Hill
Implementation
• The leading CRMs, like ERPs are
highly configurable
• Also like ERPs they are backed by a
transactional database
• Both generate large quantities of
SQL on the fly
• ERPs are starting to absorb CRMs
Copyright © 1998-2016 Curt Hill
Finally
• The type or size of businesses that
would be architected will likely have
all of these
• They will also have serious
customizations to these products
• Many other specialized applications
also exist
– EG. EA tools
Copyright © 1998-2016 Curt Hill