Knowledge Hub

Batch Processing is a method of handling data in large quantities by grouping it into sets, or batches, and processing those sets together at scheduled intervals. Unlike real-time processing, where data is handled immediately, batch processing involves accumulating data over a period and then processing it as a group.

Think of batch processing like doing your homework assignments at the end of the day rather than right after each class. You collect all the tasks and work on them together. Similarly, in batch processing, data is gathered over time, and then, as a batch, it's processed all at once.

Few examples for better understanding.

• Grade Processing in Schools: Teachers collect students' assignments, quizzes, and tests over a week and then enter grades into the system at the end of the grading period. For example, instead of entering each student's grade immediately after they submit an assignment, the teacher waits and processes grades for the entire class at once.
• Bank Transactions: Banks often update account balances and transactions overnight, processing all the transactions that occurred during the day in a batch. For example, if you deposit money or make a purchase, the transaction might not immediately reflect in your account balance; it's updated in a nightly batch.
• Payroll Processing: Companies often process payroll in batches, calculating and distributing salaries at specific intervals. For example, employee work hours, overtime, and other payroll-related information are collected over a pay period and processed together to generate paychecks.
• Inventory Management: Retailers may update their inventory levels in batches rather than immediately after each sale. For example, Sales data is collected throughout the day, and inventory levels are adjusted during a nightly batch process.
• Billing and Invoicing: Utility companies may process billing information in batches, generating bills for all customers at specific intervals. For example, Instead of generating a bill for each customer as soon as they use a service, bills are created in a batch at the end of the billing cycle.
• Batch Photo Editing: When editing photos, a batch processing approach may be used to apply the same set of edits to multiple photos simultaneously. For example, if you want to resize, crop, or apply a filter to a set of pictures, you can process them all together in a batch.
• Report Generation: Organizations often generate reports based on collected data at specific intervals, such as daily or monthly. For example, Sales reports, customer analytics, or website traffic reports may be generated in batches to provide a comprehensive overview.

Web Service is a service offered by an application to enable communication and data exchange at an application-level.

There are mainly two types of webservices used when integrating applications: REST and SOAP.

Web services are widely used in Data Integration. They allow real-time (or near-real-time) integrations and are suitable for use over the internet and in hybrid architectures (Cloud, SaaS, or on-premises).

In terms of architecture, web services enable the implementation of SOA (Service-oriented Architecture) and WOA (Web-oriented Architecture).

REST Web Services

REST, or Representational State Transfer, is a software architecture style for designing and communicating web applications.

A web service (or API) that follows REST principles is referred to as RESTful API. In the following sections, the terms REST web service and RESTful API will be used interchangeably.

A REST web service allows you to access a resource (data object, entity, etc.) via a URL (Uniform Resource Locator) [or web address].