🗊Презентация Data Management: Warehousing, Analyzing, Mining, and Visualization

Goals Recognize the importance of data, their issues, and their life cycle. Describe the sources of data, their collection, and quality issues. Describe document management systems. Explain the operation of data warehousing and its role in decision support. Describe information and knowledge discovery and business intelligence. Understand the power and benefits of data mining. Describe data presentation methods and geoinfosystems and virtual reality as decision support tools. Discuss the role of marketing databases Recognize the role of the Web in data management

Data Мanagement The amount of data increases exponentially with time. Data are dispersed throughout different organizations. Data are collected by many individuals using several methods. External data needs to be considered in making organizational decisions. Data security, quality, and integrity are critical factors of data management procedures.

Data Life Cycle Process New data collection occurs from various sources. It is temporarily stored in a database then preprocessed to fit the format of the organizations data warehouse or data marts Users then access the warehouse or data mart and take a copy of the needed data for analysis. Analysis (looking for patterns) is done with Data analysis tools Data mining tools

Data Life Cycle Continued

Data Sources Internal Data Sources are usually stored in the corporate database and are about people, products, services, and processes. Personal Data is documentation on the expertise of corporate employees usually maintained by the employee. It can take the form of: estimates of sales opinions about competitors business rules Procedures Etc. External Data Sources range from commercial databases to Government reports. Internet Databases and Commercial Database Services are accessible through the Internet.

Methods to collect Raw Data  Collection can take place in the field from individuals via manually methods time studies Surveys Observations contributions from experts using instruments and sensors Transaction processing systems (TPS) via electronic transfer from a web site

Methods for managing data collection A Data Flow Manager consists of a decision support system a central data request processor a data integrity component links to external data suppliers the processes used by the external data suppliers.

Data Quality and Integrity Internal DQ: Accuracy, objectivity, believability, and reputation. Accessibility DQ: Accessibility and access security. Contextual DQ: Relevancy, value added, timeliness, completeness, amount of data. Representation DQ: Interpretability, ease of understanding, representation

Document Management Maintaining paper documents, requires that: Everyone have the current version An update schedule should be determined Security be provided for the document The documents be distributed to the appropriate individuals in a timely manner

Transactional vs. Analytical Data Processing

The Data Warehouse Benefits of a data warehouse are: The ability to reach data quickly, since they are located in one place The ability to reach data easily and frequently by end users with Web browsers. Characteristics of data warehousing are: Organization. Data are organized by subject Consistency. In the warehouse data will be coded in a consistent manner.

The Data Warehouse Continued Characteristics of data warehousing: Time variant. The data are kept for many years so they can be used for trends, forecasting, and comparisons over time. Relational. Typically the data warehouse uses a relational structure. Client/server. The data warehouse uses the client/server architecture mainly to provide the end user an easy access to its data. Web-based. Data warehouses are designed to provide an efficient computing environment for Web-based applications

The Data Warehouse Continued

The Data Mart There are two major types of data marts: Replicated (dependent) data marts are small subsets of the data warehouse. In such cases one replicates some subset of the data warehouse into smaller data marts, each of which is dedicated to a certain functional area. Stand-alone data marts. A company can have one or more independent data marts without having a data warehouse. Typical data marts are for marketing, finance, and engineering applications.

The Data Cube One intersection might be the quantities of a product sold by specific retail locations during certain time periods. Another matrix might be Sales volume by department, by day, by month, by year for a specific region Cubes provide faster the following opportunities for analysis : Queries Slices and Dices of the information Rollups Drill Downs

Operational Data Stores It is typically used for short-term decisions that require time sensitive data analysis It logically falls between the operational data in legacy systems and the data warehouse. It provides detail as opposed to summary data. It is optimized for frequent access It provides faster response times.

Business Intelligence Business intelligence includes: outputs such as financial modeling and budgeting resource allocation coupons and sales promotions Seasonality trends Benchmarking (business performance) competitive intelligence.

Business Intelligence Continued

Knowledge Discovery KDD supported by three techniques : massive data collection powerful multiprocessor computing data mining and other algorithms processing. KDD primarily employs three tools for information discovery: Traditional query languages (SQL, …) OLAP Data mining

Knowledge Discovery Continued

Queries User requests are stated in a query language and the results are subsets of the relationship : Sales by department by customer type for specific period Weather conditions for specific date Sales by day of week …

Online Analytical Processing ROLAP (Relational OLAP) is an OLAP database implemented on top of an existing relational database. The multidimensional view is created each time for the user. MOLAP (Multidimensional OLAP) is a specialized multidimensional data store such as a Data Cube. The multidimensional view is physically stored in specialize data files.

Data Mining Data mining technology can generate new business opportunities by providing: Automated prediction of trends and behaviors. Automated discovery of previously unknown or hidden patterns. Data mining tools can be combined with: Spreadsheets Other end-user software development tools Data mining creates a data cube then extracts data

Data Mining Techniques Case-based reasoning. uses historical cases to recognize patterns Neural computing is a machine learning approach which examines historical data for patterns. Intelligent agents retrieving information from the Internet or from intranet-based databases . Association analysis uses a specialized set of algorithms that sort through large data sets and express statistical rules among items. Decision trees Genetic algorithms Nearest-neighbor method

Data Mining Tasks Classification. Infers the defining characteristics of a certain group. Clustering. Identifies groups of items that share a particular characteristic. Clustering differs from classification in that no predefining characteristic is given. Association. Identifies relationships between events that occur at one time. Sequencing. Identifies relationships that exist over a period of time. Forecasting. Estimates future values based on patterns within large sets of data. Regression. Maps a data item to a prediction variable. Time Series analysis examines a value as it varies over time.

Data Visualization Multidimensional visualization means that modern data and information may have several dimensions. Dimensions: Products Salespeople Market segments Business units Geographical locations Distribution channels Countries Industries

Data Visualization Continued Measures: Money Sales volume Head count Inventory profit Actual versus forecasted results. Time: Daily Weekly Monthly Quarterly Yearly.

Data Visualization Continued

Data Visualization Continued A geographical information system (GIS) is a computer-based system for capturing, storing, checking, integrating, manipulating, and displaying data using digitized maps. Every record or digital object has an identified geographical location. It employs spatially oriented databases. Visual interactive modeling (VIM) uses computer graphic displays to represent the impact of different management or operational decisions on objectives such as profit or market share. Virtual reality (VR) is interactive, computer-generated, three-dimensional graphics delivered to the user. These artificial sensory cues cause the user to “believe” that what they are doing is real.

Specialized Databases Marketing transaction database (MTD) combines many of the characteristics of the current databases and marketing data sources into a new database that allows marketers to engage in real-time personalization and target every interaction with customers Interactive capability an interactive transaction occurs with the customer exchanging information and updating the database in real time, as opposed to the periodic (weekly, monthly, or quarterly) updates of classical warehouses and marts.

Web-based Data Management Systems Enterprise BI suites and Corporate Portals integrate query, reporting, OLAP, and other tools Intelligent Data Warehouse Web-based Systems employ a search engine for specific applications which can improve the operation of a data warehouse Clickstream Data Warehouse occur inside the Web environment, when customers visit a Web site.

Web-based Data Management Systems

Web-based Data Management Systems

Thank you ! Questions ?