Data Engineer Job at Novel Technology Services, San Francisco, CA

a1FMd3JoSC9GeU11aXduT1pUQStwQmdPYUE9PQ==
  • Novel Technology Services
  • San Francisco, CA

Job Description

Before reading the Job description please note:

  • This position is a hybrid role and candidates must be currently located in the San Francisco Bay area.
  • The client requires candidates to possess five years of experience as a Data Engineer. While internships are valuable, they will be considered supplementary to the required five years of experience.
  • Recent work in a startup organization

Data Engineer:

Our client is Delivering the most comprehensive identity insights, our client’s platform equips businesses with fully automated KYB (Know your Business) solutions for Risk, and Fraud management, setting new standards in business verification.

The solution is designed for FS institutions to emphasize their interest in a coordinated effort to mitigate B2B fraud, reduce the risk associated with working with small businesses, and create a centralized, privacy-compliant entity for data-sharing between financial institutions.

Looking for a data engineer to assist with data collection across a variety of fragmented data sources available from both government, public, and private databases.

Responsibilities:

  • Designing the database schema
  • Migrating production tables
  • Working with large scale data in the terabyte range
  • Maintaining operational efficiency of database
  • Normalizing disparate schemas to a single unified scheme
  • Abstracting reusable components 
  • Strength in minimizing the amount of new code for new pipelines and instead creating internal packages that allow a high level of reusability 

Requirements

Experience with:

  • ETL (extract, transform, load)
  • Database design
  • Primary key, foreign key
  • Indexing
  • Partitioning
  • Access patterns
  • Migrations
  • Data pipelining
  • Core data concepts: ACID transactions, Idempotency, Orchestration

Technologies:

  • Airflow
  • Google Cloud Platform (GCP)
  • GCP Dataflow (aka Apache Beam)
  • PostgreSQL
  • Python
  • Pydantic (a python library)
  • Distributed systems

Job Tags

Full time,

Similar Jobs

ContacLink Solutions

In-Person Russian Freelance Interpreter Job at ContacLink Solutions

Language: Russian US-Based Interpreter Location: Snohomish, Washington, USA We are currently looking for both In-Person US-Based Russian interpreter for Snohomish, Washington. An In-Person Russian interpreter will provide interpretation services in a variety of settings... 

Egon Zehnder

Board and CEO Practice, Executive Search Researcher Job at Egon Zehnder

 ...internal client teams of consultants and other Experts in the execution of Executive Search, Board, and Advisory projects for multiple clients within...  ...and geographies. As a trusted adviser to leading company boards, the Practice connects leaders to the most advanced... 

Forte Behavioral Health LLC

Registered Behavior Technician - Norco Job at Forte Behavioral Health LLC

 ...Position: Registered Behavior Technician Location: Norco, CA Forte Behavioral Health LLC is committed to fostering positive change in the lives of children with developmental disabilities. Our mission is to empower children to embrace their unique learning capabilities... 

Jacuzzi Group

Director Supply Chain Job at Jacuzzi Group

 ...The Director, Supply Chain is responsible for overseeing and optimizing the end-to-end supply chain operations across Jacuzzi Groups bath manufacturing entities, with a primary focus on driving cost savings, efficiency, and strategic sourcing. This role will lead the... 

KBR

Aerospace Turbine Engineer Job at KBR

Title:Aerospace Turbine EngineerBelong, Connect, Grow, with KBR! Program Summary KBR's Army PBL program is a comprehensive logistics initiative aimed at enhancing the operational readiness of the U.S. Army while minimizing total ownership costs. Through a performance...