A global H3 population dataset optimized for consumption in Helium mapping projects.

Overview

H3 Population Helium

A global population dataset based on Uber's H3 mapping system and seeded from the open source Kontur Population dataset.

Optimized for easy consumption in Helium mapping projects.

Using the data

Download the latest kontur_population_20211109.csv.zip from this repository and extract it to your working directory.

The CSV file contains no header and two columns:

  • hex_res8 (string): H3 index at resolution 8 truncated to first 10 chars. For example, the hex 8828309565fffff will be 8828309565. If you need the full H3 index for any hex, simply append fffff.
  • population (integer): Total human population in this hex

The file is hundreds of megabytes, so it's recommended to import it into a database like Postgres for use. You can find an example of how to import a CSV into Postgres in this StackOverflow answer.

Preparing Kontur Population dataset in PostgreSQL from scratch

Although the Kontur Population dataset uses H3 resolution 8 to segment population data, it doesn't provide the H3 index of each hex. Instead, it provides a set of points that make up each hex.

We need the H3 index of each hex to easily join and compare with data in the Helium ecosystem. Let's calculate the H3 indices ourselves!

Requirements

  • PostgreSQL 9 or higher with PostGIS extension installed
  • ogr2ogr CLI (instructions below)

1. Download and extract

Download the latest dataset from this page.

The file will be of the format .gpkg.gz. Extract the .gpkg file to your working directory.

2. Use ogr2ogr to import gpkg into Postgres

You'll need ogr2ogr CLI installed for this.

If you're using Homebrew on macOS, the following command will accomplish this:

brew install gdal

Then, run the following command to import into your Postgres DB. This assumes you're running Postgres on localhost and that your DB name is heliumpop.

ogr2ogr -f PostgreSQL "PG:dbname=heliumpop" kontur_population_20211109.gpkg

This will import the data into a table called kontur_population. NOTE: If the import is stopped partway through, make sure to TRUNCATE this table before restarting the import or you will get duplicate data.

3. Convert geometry to SRID 4326 and find centers

The dataset includes hexagons in the format of PostGIS Geometry objects with SRID 3857 (Spherical Mercator). Since we're looking to get the H3 index of each hex, we need to convert the hex geometry to lat/long (SRID 4326) and find the center.

Open psql or your favorite Postgres client and run the following queries:

alter table kontur_population
add column center geometry(Geometry,4326);

update kontur_population
set center = st_centroid(st_transform(geom,4326));

4. Add H3 index to dataset

We're almost there! Add a new column for the H3 index of each hex and create a unique index to ensure consistency.

alter table kontur_population
add column hex_res8 text;

create unique index on kontur_population (hex_res8);

Clone this repo, then install required dependencies for the next step:

npm install

Copy .env.sample to a new file called .env and add your Postgres database credentials.

Then, run the H3 index update script. This will add H3 indexes, truncated to the first 10 significant characters, to the hex_res8 column of the kontur_population table:

npm run update-h3-index

5. Use or export data

And that's it! You can create the final dataset using the following query and export to CSV using your favorite tool:

select hex_res8, population
from kontur_population
You might also like...

🌀 The Javacript framework for creating a portal to your data. Perfect for a single dataset or a full catalog.

🌀 The Javacript framework for creating a portal to your data. Perfect for a single dataset or a full catalog.

🌀 Portal.JS The javascript framework for data portals 🌀 portal is a framework for rapidly building rich data portal frontends using a modern fronten

Dec 30, 2022

A homebridge plugin to display the current consumption from your Tibber pulse as lux in Home kit.

homebridge-tibber-lightsensor Tibber lightsensor is a plugin for Homebridge. It fetches the current watt usage of your house and displays it as a ligh

Oct 11, 2022

An IoT bottle that tracks water consumption. Winner of Best Health Hack, MLH's Best Hardware Hack, and QB3's Best Big Data for the Improvement of Health Care Winner at CruzHacks 2022.

An IoT bottle that tracks water consumption. Winner of Best Health Hack, MLH's Best Hardware Hack, and QB3's Best Big Data for the Improvement of Health Care Winner at CruzHacks 2022.

An IoT bottle that tracks water consumption. Winner of Best Health Hack, MLH's Best Hardware Hack, and QB3's Best Big Data for the Improvement of Health Care Winner at CruzHacks 2022.

Jan 21, 2022

Open-source dataset mapper

RDM Dataset Mapper RDM is an open-source dataset mapper. It intends to help developers transferring data from external sources into their database. ⚠️

Jun 9, 2022

📃 A public dataset of crypto addresses labeled

📃 A public dataset of crypto addresses labeled

EVM Labels A public dataset of crypto addresses labeled (Ethereum and more) Ethereum Label CSV JSON Updated exchange (Centralized Exchanges) View CSV

Jan 7, 2023

API para o Desafio 2.1 - Consumindo um dataset de filmes

A progressive Node.js framework for building efficient and scalable server-side applications. Description Nest framework TypeScript starter repository

Sep 14, 2022

A simple web server exposing Hetzner cloud instances for consumption by the Prometheus HTTP service discovery.

Prometheus: Hetzner Service Discovery A server to provide automatic node discovery for Hetzner Cloud to Prometheus via HTTP service discovery. In cont

Oct 10, 2022

Object Relational Mapping

Object Relational Mapping This package is not actively maintained If you're starting a new project, please consider using one of the following instead

Dec 14, 2022

Collect and generate mapping from file-based routers

Collect and generate mapping from file-based routers

roullector: route collector Collect and generate route data from a file-based router such as svelte-kit's What this does: show / hide !-- before --

Apr 9, 2022

A typescript data mapping tool. To support mutual transforming between domain model and orm entity.

ts-data-mapper A typescript mapping tool supports mutual transforming between domain model and orm entity. In most case, domain model is not fully com

Mar 26, 2022

PEARL (Planetary Computer Land Cover Mapping) Platform API and Infrastructure

PEARL (Planetary Computer Land Cover Mapping) Platform API and Infrastructure

PEARL API & Infrastructure PEARL is a landcover mapping platform that uses human in the loop machine learning approach. This repository contains the A

Dec 23, 2022

The Pinia plugin to enable Object-Relational Mapping access to the Pinia Store.

The Pinia plugin to enable Object-Relational Mapping access to the Pinia Store.

pinia-orm Intuitive, type safe and flexible ORM for Pinia based on Vuex ORM Next ✨ Release Notes 📖 Documentation Help me keep working on this project

Dec 31, 2022

Mind Mapping to excel, or excel to .xmind file

Mind Mapping To Excel Project setup Prepare project npm install 1、Fetch data and generate excel Open the Mind Mapping like this Process On Mind Mappi

May 5, 2022

Userland module that implements the module path mapping that Node.js does with "exports" in package.json

exports-map Userland module that implements the module path mapping that Node.js does with "exports" in package.json npm install exports-map Usage co

May 31, 2022

NoExGen is a node.js express application generator with modern folder structure, namespace/project mapping and much more! It contains preconfigured Settings and Routing files, ready to be used in any project.

Installation $ npm install -g noexgen Quick Start You can use Node Package Execution to create your node-express application as shown below: Create th

Oct 8, 2022

A library optimized for concise and principled data graphics and layouts.

A library optimized for concise and principled data graphics and layouts.

MetricsGraphics is a library built for visualizing and laying out time-series data. At around 15kB (gzipped), it provides a simple way to produce comm

Dec 22, 2022

An upgradable boilerplate for Progressive web applications (PWA) with server side rendering, build with SEO in mind and achieving max page speed and optimized user experience.

An upgradable boilerplate for Progressive web applications (PWA) with server side rendering, build with SEO in mind and achieving max page speed and optimized user experience.

React PWA v2 A highly scalable, Progressive Web Application foundation,boilerplate, with the best Developer Experience. Demo: https://demo.reactpwa.co

Dec 26, 2022

A library optimized for concise and principled data graphics and layouts.

A library optimized for concise and principled data graphics and layouts.

MetricsGraphics is a library built for visualizing and laying out time-series data. At around 15kB (gzipped), it provides a simple way to produce comm

Dec 22, 2022

🌍📖 A readable, automated, and optimized (5 kb) internationalization for JavaScript

Linguijs 🌍 📖 A readable, automated, and optimized (5 kb) internationalization for JavaScript Documentation · Documentation 2.x · Quickstart · Exampl

Jan 2, 2023
Owner
Arman Dezfuli-Arjomandi
Full-stack engineer. Host of The Hotspot podcast. older projects: falcross, jake & amir, slide game
Arman Dezfuli-Arjomandi
A library optimized for concise and principled data graphics and layouts.

MetricsGraphics is a library built for visualizing and laying out time-series data. At around 15kB (gzipped), it provides a simple way to produce comm

Metrics Graphics 7.5k Dec 22, 2022
A library optimized for concise and principled data graphics and layouts.

MetricsGraphics is a library built for visualizing and laying out time-series data. At around 15kB (gzipped), it provides a simple way to produce comm

Metrics Graphics 7.5k Dec 22, 2022
Free Bootstrap 5 Admin and Dashboard Template that comes with all essential dashboard components, elements, charts, graph and application pages. Download now for free and use with personal or commercial projects.

PlainAdmin - Free Bootstrap 5 Dashboard Template PlainAdmin is a free and open-source Bootstrap 5 admin and dashboard template that comes with - all e

PlainAdmin 238 Dec 31, 2022
The code base that powered India in Pixels' YouTube channel for more than 2 years - now open sourced for you to use on your own projects

India in Pixels Bar Chart Racing For over two years, this nifty code base powered India in Pixels' YouTube channel with videos fetching over millions

India in Pixels 141 Dec 4, 2022
A web application to search all the different countries in the world and get details about them which can include languages, currencies, population, domain e.t.c This application is built with CSS, React, Redux-Toolkit and React-Router.

A web application to search all the different countries in the world and get details about them which can include languages, currencies, population, domain e.t.c This application is built with CSS, React, Redux-Toolkit and React-Router. It also includes a theme switcher from light to dark mode.

Franklin Okolie 4 Jun 5, 2022
Fetching data from REST COUNTRIES API, this app (mobile version for now) gives information like area, population, capital, and borders for 195 countries from seven continents.

Space Travellers' Hub World Countries App works with an API which returns informations about 195 countries. Fetching data from REST COUNTRIES API, thi

Nicolae Pop 7 Aug 8, 2022
Grupprojekt för kurserna 'Javascript med Ramverk' och 'Agil Utveckling'

JavaScript-med-Ramverk-Laboration-3 Grupprojektet för kurserna Javascript med Ramverk och Agil Utveckling. Utvecklingsguide För information om hur utv

Svante Jonsson IT-Högskolan 3 May 18, 2022
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
A jQuery plugin for easy consumption of RESTful APIs

jQuery REST Client v1.0.1 Summary A jQuery plugin for easy consumption of RESTful APIs Downloads Development Version Production Version File Size Repo

Jaime Pillora 618 Nov 20, 2022