Data Systems Engineer, Rust

Posted 15 Days Ago
40 Locations
Remote
Entry level
Information Technology • Software
The Role
As a Data Systems Engineer, you will develop key technology for processing blockchain data, implement storage strategies, optimize performance in multi-tenant systems, and build data transformation pipelines. You will also leverage Apache DataFusion and contribute to the engineering of core components of The Graph.
Summary Generated by Built In

Edge & Node is at the forefront of web3 innovation. Our mission is to establish The Graph as the unbreakable foundation of open data. Our pioneering subgraphs set the industry standard and solidify The Graph as the premier solution for organizing and accessing blockchain data.

At Edge & Node, we champion a decentralized future based on shared values. Dedicated to decentralizing power and resisting censorship, we aim for a robust, permissionless information era free from central control, thus eliminating the traditional vulnerabilities associated with misplaced trust.

The Graph Network’s talented Engineering team builds and maintains core software components that power The Graph. As a Data Systems Engineer, you will do in-depth work to develop key technology that enables The Graph and participate in a growing ecosystem.

For this role, we are looking for a highly motivated and dedicated Rust Engineer who will support the development of state-of-the-art data services for processing blockchain data. If you’re passionate but don’t have experience with the specific technologies or web3, we encourage you to still apply. We value merit and skill and welcome those willing to learn and grow. 

What You’ll Be Doing

  • Leveraging and contributing to Apache DataFusion, closely monitoring upstream developments and becoming familiar with its codebase.
  • Implementing storage strategies for blockchain and other web3 data sources, while leveraging industry standard formats and keeping in mind their tradeoffs.
  • Developing the internals of a streaming processing system. Given the streaming nature of blockchains, you must understand incremental techniques to update outputs with each new block.
  • Applying resource isolation in a multi-tenant environment for resources such as CPU, RAM and storage.
  • Building data transformation pipelines specifically for blockchain data and implementing operations like aggregations and joins for streamlined usage. 
  • Optimizing compaction and garbage collection processes for performance and resource efficiency particularly in multi-tenant and high-throughput database systems. 
  • Developing supportive tooling, including tracing and  profiling.

What We Expect

  • Experience in database system internals.
  • Rust familiarity is highly desirable.
  • Knowledge of Apache DataFusion and the related ecosystem is highly desirable. (Apache DataFusion Project Page and GitHub)
  • General knowledge of distributed data systems and resource management is highly desirable.
  • Background with data ingestion from blockchain or other high-throughput sources is a plus. 
  • Adaptability to a dynamic and fully-remote work environment.
  • Rigorous approach to software development that reflects a commitment to excellence. 

Compensation/Culture

The overall market range for roles at Edge & Node is typically $100k - $500k annually. This market range is based on total compensation (vs. only base salary). Edge & Node has a culture that values meritocracy, personal growth, and an ego-free environment.

About The Graph

The Graph is the indexing and query layer of web3. The Graph Network’s self service experience for developers launched in July 2021. Developers build and publish open APIs, called subgraphs, that applications can query using GraphQL. The Graph supports indexing data from multiple different networks including Ethereum, NEAR, Arbitrium, Optimism, Polygon, Avalanche, Celo, Fantom, Moonbeam, IPFS, and PoA with more networks coming soon. To date, tens-of-thousands of subgraphs have been deployed on the hosted service, and now subgraphs can be deployed directly on the network. Over 28,000 developers have built subgraphs for applications such as Uniswap, Synthetix, KnownOrigin, Art Blocks, Balancer, Livepeer, DAOstack, Audius, Decentraland, and many others.

If you are a developer building an application or web3 application, you can use subgraphs for indexing and querying data from blockchains. The Graph allows applications to efficiently and performantly present data in a UI and allows other developers to use your subgraph too! You can deploy a subgraph to the network using the newly launched Subgraph Studio or query existing subgraphs that are in the Graph Explorer. The Graph would love to welcome you to be Indexers, Curators and/or Delegators on The Graph’s mainnet. Join The Graph community by introducing yourself in The Graph Discord for technical discussions, join The Graph’s Telegram chat, and follow The Graph on Twitter, LinkedIn, Instagram, Facebook, Reddit, and Medium! The Graph’s developers and members of the community are always eager to chat with you, and The Graph ecosystem has a growing community of developers who support each other.

The Graph Foundation oversees The Graph Network. The Graph Foundation is overseen by the Technical Council. Edge & Node, StreamingFast, Messari, Semiotic  and The Guild are five of the many organizations within The Graph ecosystem.

Top Skills

Rust
The Company
HQ: San Francisco, CA
61 Employees
On-site Workplace
Year Founded: 2021

What We Do

The initial team behind The Graph. Building a vibrant decentralized future.

Similar Jobs

Sanity.io Logo Sanity.io

Senior Software Engineer (Frontend) - Core UI

Artificial Intelligence • Enterprise Web • Software
Remote
28 Locations
190 Employees

RapDev Logo RapDev

Senior ServiceNow Developer, EMEA

Information Technology • Productivity • Professional Services • Software
Easy Apply
Remote
27 Locations
90 Employees

Smartcat Logo Smartcat

Senior ML Engineer

Artificial Intelligence • Machine Learning • Natural Language Processing • Conversational AI
Easy Apply
Remote
28 Locations
242 Employees

Sanity.io Logo Sanity.io

Senior Site Reliability Engineer

Artificial Intelligence • Enterprise Web • Software
Remote
28 Locations
190 Employees

Similar Companies Hiring

InCommodities Thumbnail
Renewable Energy • Machine Learning • Information Technology • Energy • Automation • Analytics
Austin, TX
234 Employees
RunPod Thumbnail
Software • Infrastructure as a Service (IaaS) • Cloud • Artificial Intelligence
Charlotte, North Carolina
53 Employees
Hedra Thumbnail
Software • News + Entertainment • Marketing Tech • Generative AI • Enterprise Web • Digital Media • Consumer Web
San Francisco, CA
14 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account