FHIR Data Pipes
Stay organized with collections
Save and categorize content based on your preferences.
View source on GitHub
FHIR Data Pipes is a set of ETL pipelines for transforming FHIR data into an
SQL-capable format for building analytics services. FHIR Data Pipes is designed
with horizontal scalability in mind and offers multiple deployment options.
The features include:
Apache Beam ETL pipelines to transform
data from a FHIR source into an SQL-on-FHIR schema data warehouse, based on
Apache Parquet files that can be queried using SQL
(see Parquet-on-FHIR schema)
Controller Module for managing the FHIR Data Pipes Pipelines, integrating
"full", "incremental", and "merger" pipelines together. Using the controller
module you can schedule periodic incremental updates or use the Web Control
Panel to start the pipeline manually.
Implementation of the SQL-on-FHIR-v2 specification
making it possible to apply ViewDefinition resources to generate flat views
within the pipelines (see View layer)
Go to developer documentation
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-11-26 UTC.
[null,null,["Last updated 2024-11-26 UTC."],[[["\u003cp\u003eFHIR Data Pipes is a scalable ETL toolset that transforms FHIR data into an SQL-compatible format for analytics.\u003c/p\u003e\n"],["\u003cp\u003eIt utilizes Apache Beam pipelines to create a data warehouse based on an SQL-on-FHIR schema using Parquet files.\u003c/p\u003e\n"],["\u003cp\u003eA controller module allows for managing and scheduling pipeline updates, including full, incremental, and merging processes.\u003c/p\u003e\n"],["\u003cp\u003eFHIR Data Pipes supports the SQL-on-FHIR-v2 specification for generating simplified views of the data.\u003c/p\u003e\n"]]],["FHIR Data Pipes provides ETL pipelines to convert FHIR data into an SQL-ready format using Apache Beam. These pipelines create a data warehouse based on Apache Parquet files, enabling SQL queries. A Controller Module manages \"full,\" \"incremental,\" and \"merger\" pipelines, allowing scheduled or manual updates. The system implements the SQL-on-FHIR-v2 specification, supporting ViewDefinition resources to create flat views within the pipelines. It offers scalability and multiple deployment options for analytics services.\n"],null,["# FHIR Data Pipes\n\n[View source on GitHub](https://github.com/google/fhir-data-pipes)\n\n*FHIR Data Pipes* is a set of ETL pipelines for transforming FHIR data into an\nSQL-capable format for building analytics services. FHIR Data Pipes is designed\nwith horizontal scalability in mind and offers multiple deployment options.\n\nThe features include:\n\n- [Apache Beam](https://beam.apache.org/) ETL pipelines to transform\n data from a FHIR source into an SQL-on-FHIR schema data warehouse, based on\n Apache Parquet files that can be queried using SQL\n (see [Parquet-on-FHIR schema](/open-health-stack/fhir-analytics/parquet-on-fhir))\n\n- Controller Module for managing the FHIR Data Pipes Pipelines, integrating\n \"full\", \"incremental\", and \"merger\" pipelines together. Using the controller\n module you can schedule periodic incremental updates or use the Web Control\n Panel to start the pipeline manually.\n\n- Implementation of the [SQL-on-FHIR-v2 specification](https://build.fhir.org/ig/FHIR/sql-on-fhir-v2/StructureDefinition-ViewDefinition.html)\n making it possible to apply ViewDefinition resources to generate flat views\n within the pipelines (see [View layer](/open-health-stack/fhir-analytics/view-layer))\n\nGo to [developer documentation](https://google.github.io/fhir-data-pipes/)"]]
