Python ELT Studio, an application for building ELT (and ETL) data flows.

Related tags

Database pelt-studio
Overview

PELT Studio

The Python Extract, Load, Transform Studio is an application for performing ELT (and ETL) tasks. Under the hood the application consists of a two parts. The Python back end: using SQLAlchemy for interfacing with traditional Relational Databases, FastAPI for exposing the logic to the fronend and Motor (async MongoDB client) for persisting the application configurations and database metadata. The front end is built using React and React-Flow from the Databricks project.

PELT Studio example ELT dataflow

The above diagram serves as a basic explaination of what the MVC for this app strives to achieve. Current progress is shown below:

PELT Studio pre alpha interface

Usage

# clone the repo and cd into it
git clone https://github.com/schlerp/pelt-studio.git
cd pelt-studio

# set up the python environment
python -m venv venv
source ./venv/bin/activate
python -m pip install -r requirements.txt

# start the app
python -m peltapi

# frontend: http://localhost:8321/peltstudio
# backend docs: http://localhost:8321/docs
You might also like...

A typescript data mapping tool. To support mutual transforming between domain model and orm entity.

ts-data-mapper A typescript mapping tool supports mutual transforming between domain model and orm entity. In most case, domain model is not fully com

Mar 26, 2022

A JSON Database that saves your Json data in a file and makes it easy for you to perform CRUD operations.

What is dbcopycat A JSON Database that saves your Json data in a file and makes it easy for you to perform CRUD operations. ⚡️ Abilities Creates the f

Jan 8, 2023

Fast File is a quick and easy-to-use library to convert data sources to a variety of options.

Fast File is a quick and easy-to-use library to convert data sources to a variety of options.

Fast File Converter The Express.js's Fast File Converter Library Fast File Converter Library is a quick and easy-to-use library to convert data source

Nov 16, 2022

E-Commerce Application developed with nodejs and stored to mongodb.

E-Commerce Application This Application has been developed with nodejs and mongodb. Environment Variables Create a file named config.env in config dir

Dec 23, 2021

Database manager for MySQL, PostgreSQL, SQL Server, MongoDB, SQLite and others. Runs under Windows, Linux, Mac or as web application

Database manager for MySQL, PostgreSQL, SQL Server, MongoDB, SQLite and others. Runs under Windows, Linux, Mac or as web application

Database manager for MySQL, PostgreSQL, SQL Server, MongoDB, SQLite and others. Runs under Windows, Linux, Mac or as web application

Dec 30, 2022

Simple CRUD application with Nestjs, Prisma and Docker with Postgres

A progressive Node.js framework for building efficient and scalable server-side applications. Description Nest framework TypeScript starter repository

Nov 6, 2022

An open source cybersecurity protocol for syncing decentralized graph data.

An open source cybersecurity protocol for syncing decentralized graph data.

GUN is an ecosystem of tools that let you build community run and encrypted applications. Currently, Internet Archive and HackerNoon run GUN in produc

Dec 27, 2022

Composable data framework for ambitious web applications.

Orbit Orbit is a composable data framework for managing the complex needs of today's web applications. Although Orbit is primarily used as a flexible

Dec 18, 2022

:green_book: SheetJS Community Edition -- Spreadsheet Data Toolkit

:green_book: SheetJS Community Edition -- Spreadsheet Data Toolkit

SheetJS js-xlsx Parser and writer for various spreadsheet formats. Pure-JS cleanroom implementation from official specifications, related documents, a

Jan 9, 2023
Comments
  • Can't get the pelt-studio to run

    Can't get the pelt-studio to run

    Followed install instructions. Ran server. Navigated to http://localhost:8321/peltstudio and the page retuned with {"detail":"Not Found"}

    opened by clivebearmanhub 4
Owner
Schlerp
I like data analysis, machine learning, automation and systems integration.
Schlerp
GitHub
An example repository on how to start building graph applications on streaming data. Just clone and start building 💻 💪

Example Streaming App ?? ?? This repository serves as a point of reference when developing a streaming application with Memgraph and a message broker

Memgraph 40 Dec 20, 2022
AlaSQL.js - JavaScript SQL database for browser and Node.js. Handles both traditional relational tables and nested JSON data (NoSQL). Export, store, and import data from localStorage, IndexedDB, or Excel.

Please use version 1.x as prior versions has a security flaw if you use user generated data to concat your SQL strings instead of providing them as a

Andrey Gershun 6.1k Jan 9, 2023
A progressive Node.js framework for building efficient and scalable server-side applications

A light template to easily setup your backend with a simple jwt authentication with Nest.js and TypeScripit. Powered with Prisma and Hot-reload features

Junior Medehou 3 Feb 25, 2022
A progressive Node.js framework for building efficient and scalable server-side applications

A progressive Node.js framework for building efficient and scalable server-side applications. Description Nest framework TypeScript starter repository

Gustavo Lopes 3 Oct 31, 2022
TypeScript ORM for Node.js based on Data Mapper, Unit of Work and Identity Map patterns. Supports MongoDB, MySQL, MariaDB, PostgreSQL and SQLite databases.

TypeScript ORM for Node.js based on Data Mapper, Unit of Work and Identity Map patterns. Supports MongoDB, MySQL, MariaDB, PostgreSQL and SQLite datab

MikroORM 5.4k Dec 31, 2022
Add hic et nunc data into your websites and Node.js scripts

hic et nunc API Guide Build websites and Node.js scripts with hic et nunc data hic et nunc is a decentralized NFT marketplace built on the Tezos block

Ian Petrarca 34 May 3, 2022
Jsynchronous.js - Data synchronization for games and real-time web apps.

Jsynchronous.js Synchronize your rapidly changing app state with all connected browsers. Jsynchronous ensures all clients see the same data as what’s

Sirius 111 Dec 16, 2022
Nodeparse - A lightweight, vanilla replacement for Express framework when parsing the HTTP body's data or parsing the URL parameters and queries with NodeJS.

nodeparse A lightweight, vanilla replacement for Express framework when parsing the HTTP body's data or parsing the URL parameters and queries with No

Trần Quang Kha 1 Jan 8, 2022
A visualizer that parses CSV data and renders it in a table in Postman or in a browser.

Postman CSV Visualizer A visualizer that parses CSV data and renders it in a table in Postman or in a browser. Example Using the Visualizer in Postman

Philippe Ozil 18 Dec 10, 2022
Dead Simple Postgres Data Viewer and Query Runner

Dead Simple Postgres Data Viewer and Query Runner Environment Variables GITHUB_CLIENT_ID Github Client ID of the Oauth Application for protecting your

Mahesh C. Regmi 7 Aug 22, 2022