Technologies

Data Acquisition

Data acquisition overview

Before starting to build analysis in SRK, end users need to load their data into the system first. SRK provides various data acquisition mechanisms for users to do so – from record-by-record manual data entry to bulk import of large volume of data. Bulk import is the most frequently used data acquisition mechanism since it is the most efficient way to load data into SRK.

Begin with summary data

As a solution for business users to develop analytical centric reporting, SRK operates at the level of summary data, i.e., the data went into SRK is already aggregated to certain degrees per the analytical requirement such as operational reports, etc. To understand this better, think about how business users use Excel to build the analysis.

  • Originally, there are multiple spreadsheets generated in various ways – manually put together, exported from a report in the ERP system, etc.
  • Data in these spreadsheets are summary data in the sense that, business users can already use these spreadsheets as reports to present certain type of information.
  • When there is the need to get more information out of existing spreadsheets (i.e., to create more aggregated and/or complicated analysis), business users then manually combine them into one larger spreadsheet based on certain criteria. These criteria are business rules for summary data other than database relationships for raw data therefore business users can combine data using spreadsheet without having to rely on IT to write SQL.

Using SRK, instead of manually combining data from one spreadsheet into another spreadsheet, users can import summary data residing in those spreadsheets into SRK. Each spreadsheet then becomes one data series. After that, users can develop their analysis by creating views to combine data from multiple data series and/or views using the same set of business rules they used in spreadsheet, building custom computation and aggregation, etc. On top of all these, users can take further advantages SRK and build their own dashboards, use tasks and notifications to improve collaboration efficiency, and much more.

Import data from a variety of data sources

1. SRK Standard Importer

For summary data that has been stored in CSV files and/or spreadsheets, user can utilize the SRK standard importer to import it. With the standard importer, mappings can be configured between columns in CSV files/spreadsheets and fields in SRK Data Series. Multiple CSV files/spreadsheets can be scheduled to load into SRK periodically, and notification can be configured to let user know the results of import – whether it is successful or not. The standard importer also provides error recovery functionality that allows invalid data to be corrected and re- imported.

2. SRK Advanced Importer and Data Acquisition Framework

Summary data may not always be stored in CSV files or Excel spreadsheets. Therefore SRK also provides an advanced importer that allows users to import summary data into SRK from a variety of data sources. The advanced importer is built on top of the SRK data acquisition framework that allows easily addition of new/custom data source support. A large number of data sources are supported out-of-the-box. And the flexible architecture of SRK data acquisition framework enables the advanced importer to be easily extended to support user-specific data sources and data import requirements.

3. ETL Tool

For those source systems that are not able to export summary data into CSV files and/or spreadsheets or when users choose to extract either raw data or summary data by their own data extraction mechanism, they can follow the “Guidelines to Populate SRK Data Series” provided by DrivenBI and populate data directly into SRK Data Series’ with an ETL tool. The process to load data into corresponding SRK Data Series is generally less complicated than traditional ETL, as SRK is designed to perform data transformation and filtering during view creation thereby centrally manages business rules inside of SRK rather than outside as well as providing maximum flexibility.

Performance considerations

In normal use, SRK deals with gigabytes of data rather than terabytes or petabytes. That is because it is working with summary data rather than raw data. With the ever-increasing size and volumes of data sources in business, however, even summary data can become quite large. To keep pace with this anticipated growth, SRK is designed and continually refined to address performance; for example, all custom views that users create in SRK are not virtual database views but are materialized views which
reside in physical database tables. Thus when users are building analysis across multiple views a much better query performance can be achieved.

Summary

Once data is imported into SRK, users can develop their analysis by creating views to combine data from multiple data series, building custom computation and aggregation, combining views to create further aggregated views, creating dashboards to deliver interactive and real- time content navigation, using tasks and notifications to improve
collaboration efficiency, and much more.

Data Linkage & Just-in-time Analysis

Data Linkage & Just in Time Analysis Overview

Through various data acquisition mechanisms, users’ data come into SRK and reside in Data Series. Starting from there users can link data from multiple Data Series and build their analyses using Custom Views. Furthermore, Custom Views can also be built on top of other Custom Views so as to take advantages of others’ work and produce more useful and more sophisticated analyses. All these can be accomplished without the need of any IT expertise or programming skills, allowing SRK users to produce analysis at a fraction of the time and cost of other conventional BI solutions where ETL and data warehouse are inevitable.

Combining Data from Multiple Sources

Data Series is the centralized data storage in SRK. Data from various external sources are consolidated in this one place serving as the source for building analysis. Using Custom Views, users can easily link data from multiple data series and analyze them all at once thereby accomplishing the purpose of combining data from multiple sources and building useful analyses to reveal a bigger picture. More importantly, all these are done without the need of ETL or data warehouse.

Just in Time Analysis

Custom Views in SRK is a powerful yet easy to use feature for users to build their analyses, either on top of multiple Data Series or based on the Custom Views other users have built. Featuring functions such as matching field that allows user to define flexible criteria on how to combine data from the selected sources; compute field and aggregate field that support free form mathematics calculation, string manipulation and date calculation; data grouping and aggregation functions; advanced filtering and, last but not least, highlight and chart that allow users to produce useful and intuitive visualizations for their analyses.

Once a Custom View is set up, it will automatically refresh upon import of new data and/or change of existing data. This guarantees the analysis is always up-to-date. Users are also allowed to make changes to their Custom Views on the fly and they will see updated analyses right away. All these can be accomplished without the need of any IT expertise or programming skills. All in all, SRK helps business users to get their analyses just in time.

Multi-dimensional Analysis

Multi-dimensional Analysis Overview

The next generation analytical engine of SRK empowers users with the ability of building multi-dimensional analysis without the need of building a data warehouse or knowing data modeling technique. SRK brings powerful data analysis capabilities such as drilldown and rollup to the users while not requiring any IT expertise or programming skills.

Pivot View

Unlike other BI tools, the Pivot View feature of SRK allows users to build multi-dimensional analysis without having to deal with those data warehouse nuisances such as building data models, figuring out facts and dimensions as well as the relationship among them, etc. Pivot View provides an easy-to-use UI with which users can create their desired multi-dimensional analysis by simply picking the fields from SRK Data Series and/or Custom Views containing the data they want to analyze via drag and drop, and configure aggregations via point and click. Using Pivot View, users can perform drilldown, rollup, and slice and dice their data whichever way they want. Chart capability is also available on Pivot View, providing stunning and interactive visualizations. Moreover, Pivot View is seamlessly integrated with other SRK features such as Page, Report, and Task, making the already powerful analysis fully actionable.

Embracing the Power of Big Data

Behind the scene, the OLAP cube that is essential for supporting multi-dimensional analysis is dynamically generated on the fly, thereby taking away the necessity of building a Data warehouse and the demand of ETL. The analytical engine that backs all these is built on top of Hadoop – the most mature Big Data solution out there. By utilizing Hadoop technology, plus DrivenBI’s proprietary technique and fine-tuning, SRK is able to process enormous amount of data and support analysis scenarios that are hard to accomplish using relational database technology, and at the same time offers superior performance.

By embracing the power of Big Data, SRK provides not just dynamic cube generation, but also excellent scalability and a great deal of analysis possibilities.

Pages

Page overview

Page is a dashboard-like feature of SRK where users can put tabular data and/or charts of multiple views into a single place. With Page, users can configure their own dashboards to display multiple analyses in a customized layout and in their preferred visual presentation (table or chart). When it comes to displaying analyses from different perspectives (for example, by region) or at different granularities (for example, daily, monthly, and YTD), tables and charts can be organized using hierarchical menus which makes it very flexible and at the same very easy to navigate. With all these, Page allows users to produce an informative overview of their analyses. Getting the big picture at a single glance.

Customizable Content & Layout

Page allows users to display multiple views in a grid layout with customizable number of rows and columns. What to be displayed in each “cell” of the layout grid is also fully customizable – users can pick any view in the organization and put it any where on the Page. Each of those picked views can be configured to present themselves as either table or chart. This offers a very flexible layout mechanism using which users can put any views onto the Page and arrange them however they like.

Hierarchical Content Organization

Views displayed on one Page could present analyses from different perspectives (by region) or at different granularities (daily, week, YTD, etc.). In this case, Page offers hierarchical menus for better organizing these views. Users can add multiple menus to a Page, each of which can then be associated with one or multiple views. With menus on Page users would only see views associated with the menu that is currently been selected. To switch to a different set of views is as simple as selecting a different menu. Using the example above, users can create menus representing region, product, weekly time scale and so on and so forth, and then associate them with corresponding views. Menus can be organized hierarchically to clearly illustrate the different perspective/granularity of each set of views. Using menus to organize views on a hierarchical way further extends the capacity of Page and allows users to perform cross-functional analysis, getting the insights that would not be obtained easily when looking at each view independently.

Collaboration & Data Sharing

Collaboration & Data Sharing Overview

Powerful analysis translates into action. However, getting the right information to the right people has traditionally been a manual and time- consuming process. Therefore, SRK features centralized, intuitive, and actionable information sharing to facilitate coordination and collaboration among team members.

Reports

Besides just presenting information, reports in SRK are also bundled with rich collaboration features that allow team members to work together effectively.

● Sharing

Reports are meant to be shared, and the report feature in SRK is designed from the ground-up to facilitate actionable sharing among team members. SRK users can easily publish their reports to an entire organization, which would make the reports accessible for all users in that organization. Or, in the cases where more exclusive access is required, reports can be shared with designated users only.

● Comments and Annotations

A shared report serves as a centralized virtual forum for all users who have access to review, discuss, and coordinate tasks. Annotations can be added to each record in a report. Allowing for easy and effective interaction among team members. Comments are threaded to enhance collaboration and also support linking to specific records in the report. Allowing for comprehensive and detailed discussions.

● Scheduling

Users can automate their reporting process by setting up schedule for report generation. Reports can then be generated automatically at user- defined intervals – no matter it is quarterly financial reports, weekly operational reports, or daily status reports.

● Email Notifications

Email notifications will be delivered to users once their scheduled reports have been generated. Users also have the option to have the content of the report attached with emails in the format of PDF for ease of access. Email notifications in combination with scheduling features enable a completely automated reporting process for SRK users.

● Export

Reports, including users’ annotations, can be exported into PDF, XLS, or CSV format. This gives users more flexibility in terms of sharing their analysis with others and/or incorporating these reports into their
PowerPoint slides, and so on and so forth.

Tasks

SRK provides a centralized task management system to facilitate team collaboration. With tasks, team leaders can assign, monitor, and coordinate team members’ work using a user-friendly interface.

● Integration with reports

SRK users can create tasks directly from a report, which allows the task assignee to easily trace back to the exact piece of information in the report that is relevant to the task.

● Comments

Comments allow task assignee, users on the CC list, and task reporter to communicate with each other in real-time. Email notifications will be sent out when a new comment is posted to a task. Comments also serve the purpose of keeping track of all the updates to a task.

● Attachments

Tasks allow users to attach any time of files. Providing more means of sharing task related information among team members.

Security

Security Overview

SRK ensures the security of users’ data from multiple angles – data transfer, data storage, and system functionality. Offering comprehensive protection while still allowing flexible and easy access to the data.

Data Security

SRK uses HTTP (SSL) as the communication protocol thus all the data being transferred back-and-forth between client and server is encrypted. Users’ data is automatically backed up and stored in multiple distributed locations, offering extra reliability.

SRK also enforces strong user passwords via configurable password policy. Additional security measures such as periodical password expiration, if needed, can be set up as well.

Role-based Record Level Access Control

SRK supports several different types of users with which administrators can easily provision functional level permissions for each user account. For instances, power users have access to all system features and can be granted with administrative privilege; view users can access the majority of system features and have restricted access to other users’ data; report users can only subscribe to scheduled reports; etc.
More significantly, SRK offers a role-based record level access control mechanism. Administrators can defined any number of roles, set up access control criteria for each role, assign roles to users and then associate access control criteria with Custom Views so that users would only see the data they are allowed to. For example, roles and access control criteria can be set up in the way that department managers are only able to see the records pertaining to their own department, even when the Custom View actually contains records from all departments.

This role-based record level access control mechanism operates at a much finer granularity than other BI tools out there. It offers great flexibility and is capable of supporting nearly any access control scenario imaginable.

SRK Importer

SRK Advanced Importer

Among the three ways of importing data into SRK, the Advanced Importer is the most powerful and flexible one for users to import their summary or raw level data from different sources such as web services, SQL databases, PLC, and CSV files/Excel spreadsheets reside in their local computers. In addition, further transformations on their data can be performed – such as value substitution, string truncation and concatenation, date calculation. The Advanced Importer is capable of automatically populating the mapping between for example columns in CSV files/spread- sheets and fields in SRK Data Series while also allows users to do further adjustment. After installation, users can use the Advanced Importer from within the web-based UI of SRK. The Advanced Importer also provides result notification for the import and error recovery functionality that allows invalid data to be corrected and re-imported.

SRK Standard Importer

The SRK Standard Importer is specialized in handling CSV files and Excel spreadsheets to be imported into SRK. It is a Windows-based application that provides the conventional user experience which could be appreciated by certain groups of users. Similar to the SRK Advanced Importer, mappings are automatically populated and users are also allowed to do their own configuration. The results of the import – whether it is successful or not – is shown to the user during the import process and sent to the user via emails after import is finished. The Standard Importer also provides error recovery functionality that allows invalid data to be corrected and re-imported. As also its specialty, the Standard Importer is able to handle Excel spreadsheets generated by very earlier versions of Microsoft Office (e.g., Office 95).

Express Import

The Express Import is a built-in feature of SRK and requires absolutely no installation of extra application. Users would be using the web-based UI of SRK to import CSV files from their local computer or import reports from Salesforce.com. The Express Importer also supports creating new Data Series automatically based on the file/report structure that being imported therefore further saves users the effort to create the Data Series separately.