We are actively seeking a skilled and innovative Data Engineer with a specialized focus on prompt engineering. In this role, you will play a crucial part in developing and fine-tuning our natural language processing models by creating effective and diverse prompts for training. You will collaborate closely with our CTO and key stakeholders to enhance the performance and capabilities of our language models
About Us
Being an early stage, category-defining company necessitates a deep understanding of our customers’ pain points, problems to solve, and areas of opportunity - all of these things help define what is possible with OnLoop in their organizations. Our founders were tired of the time-consuming yet biased and anxiety riddled traditional approaches to performance management and manager enablement to abolish an archaic industry and replace it with something new — Collaborative Team Development (CTD)
Key Responsibilities
- Design and implement scalable, high-performance data architectures that support the storage and processing of structured and unstructured data
- Develop and maintain data models, ensuring data accuracy, consistency, and reliability
- Build and optimize ELT processes to extract, transform, and load data from various sources into the data warehouse
- Collaborate with stakeholders to understand data requirements and ensure data quality throughout the ELT process
- Create and maintain robust data pipelines for real-time and batch processing
- Implement data streaming solutions to enable timely and efficient data delivery
- Administer and optimize data warehouses to ensure performance, security, and scalability
- Implement and enforce data governance policies to ensure data quality, privacy, and compliance
- Collaborate with security teams to implement and maintain data security measures
- Work closely with cross-functional teams, including product owners, analysts, and software engineers, to understand data requirements and deliver effective solutions
- Experiment with different prompt architectures to improve model accuracy and performance
- Curate and preprocess datasets to create training data specifically tailored for prompt-based learning
- Implement data augmentation techniques to enhance prompt diversity and model robustness
- Communicate complex technical concepts to non-technical stakeholders
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Systems, or a related field
- Proven experience as a Data Engineer or similar role, with a track record of designing and implementing data solutions
- Strong proficiency in programming languages such as Python, with a focus on data manipulation and preprocessing
- Familiarity with natural language processing models and their training processes
- Experience with machine learning frameworks and libraries, such as TensorFlow or PyTorch
- Expertise in database technologies, such as PostgreSQL, BigQuery and Snowflake
- Experience with data processing frameworks such as Apache Spark, Airbyte, Airflow, DBT
- Familiarity with cloud platforms (AWS, Azure, GCP) and their data services
- Knowledge of data governance, security, and compliance best practices
- Have knowledge of source control systems such as Git
- Ability to identify bottlenecks in our data pipeline and streamline them
- Experience with any kind of prompt engineering