Software Engineer

itD is seeking a Software Engineer to design and scale the data pipelines that power next-generation foundation models for machine-generated data, including time series, logs, and large-scale event streams. This role contributes directly to the success of model training and production systems by enabling reliable, high-performance data infrastructure at scale. The ideal candidate will bring deep experience in distributed systems and data engineering, along with a proven track record of delivering scalable, production-ready data pipelines that support machine learning workflows.

Location: Remote (U.S.-based; time zone alignment with Pacific or Central preferred)

We provide comprehensive medical benefits, a 401(k) plan, paid holidays, and more. Please note that we are only considering direct W2 candidates at this time, as we are unable to offer sponsorship.

Responsibilities:
• Build and scale distributed data pipelines for large-scale time series, log data, and high-volume event streams.
• Design and maintain reliable, high-performance Spark and Python workflows to support model training datasets.
• Analyze and resolve performance bottlenecks related to latency, memory utilization, data skew, and throughput.
• Improve data quality, validation processes, and reproducibility for machine learning workloads.
• Partner with machine learning engineers and researchers to