Data Silos

Data Silos is always confused with Data Island.

Data Islands are those datastores which are not accessible from external world may be due technology barriers, but Data Silos are those which are only accessible for their relevant stakeholders.

There can be a lot of reasons why data is in silo.

• A particular datastore might be hosting confidential data e.g., finance data, audit data etc.
• A particular data which is not readable e.g., semi or unstructured data sets.
• There is Security or Privacy concerns, so data is not shared among another departments.
• The quality of data is so low that it can’t be shared with management as it may cause wrong decisions and may damage the company’s image.
• Sometime, in development environment different database instances are created for different development teams so if one changes data or table’s structures, other development teams are not impacted.

Why should Data Silos not exist?

In the current era, data driven organization concept is flourishing where not only data is brought at centralized location rather data democratization and liberalization has also been made a norm.

If data is not residing at one location, then it’s not possible to correlate the same kind of data across multiple systems. Data Scientists or top management always want cross-departmental dashboards and reports.

Data Warehouse came into being to cater for this problem where data in source systems were sitting in silos e.g., almost all medium to large organizations have a CRM system, ERP system, Procurement system, HR system etc. Now, if management want to have information from all the above-mentioned systems, then first it's very time-consuming to extract data and secondly the quality of data is also very difficult to validate. In Date Warehousing, data is brought from all kinds of source systems in one place in unified data model for have single source of truth for Decision Support Systems.