Introduction
For my second co-op work term, I worked as an Undergraduate Research Assistant with the Ontario Agricultural College at the University of Guelph. I supported the Net-Zero Dairy Genome Project, with my role focused on the software and data infrastructure behind its dairy emissions research.
The central challenge was preserving meaning from the barn to the database. A methane reading is only useful when researchers can understand when, where, and how it was collected, yet devices move, installations change, and no single file captures the full physical system behind a measurement. My work built a bridge between those physical realities and a research dataset whose origins could be traced and trusted.
The employer
The Ontario Agricultural College (OAC) at the University of Guelph brings together research in animal health, genetics, nutrition, welfare, and sustainable food production. I worked within its Department of Animal Biosciences, where computational tools are increasingly important to answering biological and agricultural questions.
At the Centre for Genetic Improvement of Livestock (CGIL), genetics, genomics, statistics, and computing come together to answer practical questions about livestock. This work draws on data from many animals, devices, farms, and research partners, making dependable software infrastructure essential to reproducible analysis.
My placement contributed to the Net-Zero Dairy Genome Project, a national collaboration studying how genetics, nutrition, and management practices can reduce the environmental impact of dairy production while supporting animal health and productivity. I worked within the project’s data effort, at the intersection of agricultural research, distributed data collection, and software and data engineering.
- Project
- Net-Zero Dairy Genome Project
- Industry
- Agricultural Genomics
- Group
- CGIL / Animal Biosciences
Job description
My role was to build software for a national data-collection network of more than 50 sniffers across 28 farms in Canada. Each sniffer could produce second-by-second gas measurements, while automated milking robots supplied animal and session information and installation records described how devices were connected to farm systems over time. The engineering challenge was to combine those separate sources into clean, analysis-ready data without losing the context needed to interpret it.
Most of my development work took place on Linux servers. I used Python to build ingestion and validation workflows, and SQL to model and query the data in PostgreSQL and TimescaleDB. Developing loaders and database migrations required me to learn relational schema design and time-series storage. Because new measurements arrived every second, I also had to consider memory use, repeated processing, and efficient database writes from the beginning.
The supporting tools each served a different purpose. R helped me explore unfamiliar datasets and check assumptions before formalizing transformations, while Bash automated routine server tasks. I used Git to track changes to the loaders, schemas, and migrations. Docker provided a consistent environment for local development and testing, whereas Apptainer allowed the workflow to run in the research server environment.
The work was not confined to a computer. During farm visits, I calibrated the physical gas-sniffing equipment and saw how the sensors, sampling lines, and automated milking robots were arranged. Understanding how the measurements were collected helped me model the equipment relationships correctly and recognize when incoming data lacked important context.
I also presented project progress, architecture, and data workflows to team members, external stakeholders, and visiting researchers. These presentations required me to move between implementation details, scientific needs, and the practical value of the system depending on the audience. The resulting discussions clarified requirements and kept the software connected to the research it was intended to support.
Technical work
The engineering work followed the dependencies in the data. Installation records first established which sniffers, sampling noses, and farm systems were connected at a given time. The sniffer and milking loaders could then use that shared registry to interpret their files before writing readings and sessions to PostgreSQL and TimescaleDB. From there, I focused on making the complete workflow safe to rerun, deploy, and maintain.
Understand the Inputs
I began with small Bash scripts that inspected representative installation, sniffer, and milking exports. They compared headers and field types, surfaced vendor differences, and showed how timestamps and missing values behaved across files. This was a practical way to learn what each source actually guaranteed before giving it a permanent database representation.
I turned those findings into documented source contracts and validation rules that fail early when an input changes. A file that does not match its expected structure is rejected at the boundary with a useful error, rather than being partially interpreted later in the pipeline.
Build the Schema and Registry
With the input boundaries understood, I defined the model through versioned SQL migrations in PostgreSQL, using TimescaleDB for high-frequency gas readings. pgAdmin helped me inspect relationships and verify loaded data. Installation records and device events populated the shared registry that both loaders use to interpret the farm equipment.
My first schema treated a robot number as the complete equipment identity. Real farm configurations broke that assumption: a farm could contain several robots, feeding stations, or both, and the number was only a local label. I redesigned the model around a general system identity connected to devices and sampling noses through validity-dated installations, supporting equipment changes without forcing everything into a robot-only structure.
Build the Loaders
I built two ingestion and validation workflows in Python. The sniffer loader validated long-format gas records, normalized timestamps to UTC, selected the applicable sampling nose, and resolved the active installation. It then used bulk database writes to store one time-series row per device and timestamp without discarding the equipment context.
The milking loader handled its configured vendor exports separately, using the same registry to resolve the farm system before storing normalized animal and session records. Keeping the loaders separate avoided forcing different sources through one parser. I used R outside the loading path for ad hoc data checks and supporting tasks.
Strict time normalizationrobot/process.py# Combine the source's separate date and time columns. local_timestamps = pd.to_datetime( date_series.astype("string").str.strip() + " " + time_series.astype("string").str.strip(), errors="raise", ) # Reject ambiguous or nonexistent local times instead of guessing. localized_timestamps = local_timestamps.dt.tz_localize( ZoneInfo(timezone), ambiguous="raise", nonexistent="raise", ) # Convert every source to one comparable time standard. return localized_timestamps.dt.tz_convert("UTC")Harden and Hand Off
After end-to-end loading worked, I designed for repeated operation rather than ideal runs. Each file is loaded in a transaction, so a failed file leaves no partial records. File fingerprints make exact reruns safe and expose changed content, while advisory locks prevent overlapping runs. I used Git to manage changes to the schema and loaders, with automated tests and continuous integration exercising the complete workflow instead of each script in isolation.
Making the system usable beyond my machine required a repeatable environment and a clear handoff. Docker supported local development and testing, while Apptainer carried the workflow onto the Linux research servers. Documentation completed the handoff for two audiences. Researchers received guidance on available data, workflow operation, and validation results. Future developers received architecture, setup, schema, migration, design-decision, and troubleshooting documentation.
Learning goals
Critical and Creative Thinking: Problem Solving
- Skills I wanted to learn
- I wanted to strengthen my ability to break down unfamiliar data problems and design solutions that were both practical and reliable. My action plan was to understand the purpose and relationships within each dataset before changing a workflow, document recurring quality issues, and seek feedback when comparing possible designs.
- Technologies and tools involved
- I applied Python, SQL, R, database design, automated validation, logging, and testing while working through these problems. Rather than treating each issue as an isolated error, I learned to consider how a decision could affect the full workflow and the researchers who depended on its output.
- Why this mattered
- This mattered because research conclusions depend on trustworthy data. A technically successful program is not enough if its assumptions are unclear or if unexpected input can produce misleading results. Strengthening this skill helped me design solutions with accuracy, traceability, and future maintenance in mind.
- Reflection
- This goal was completed. My first identity model failed against the real farm configuration because it treated a local robot number as a complete equipment identity. Recognizing the wrong assumption, revising the abstraction and validating it against the real operation made me more confident turning broad data problems into explicit models, contracts, and validation rules.
Literacy: Technological Literacy
- Skills I wanted to learn
- I wanted to become more comfortable choosing and using the tools involved in a database-backed research workflow. My action plan was to study the existing system before making changes, use the most appropriate language for each task, and make my work reproducible and testable.
- Technologies and tools involved
- I used Python and R for data processing, SQL for organizing and querying information, and Bash and Linux for automation and server-based work. Moving between these tools helped me understand how individual scripts, databases, and operating-system tasks fit together as one system.
- Why this mattered
- This mattered because research software often combines several technologies rather than relying on one application or language. Improving my technical literacy allowed me to work more independently, diagnose problems across different parts of the workflow, and make better-informed engineering decisions.
- Reflection
- This goal was completed. I moved comfortably between Python transformations, SQL schema design, TimescaleDB, Bash, Linux, continuous integration, and container tooling. More importantly, I learned how these technologies cooperate as one maintainable system rather than treating them as isolated tools.
Global Understanding: Sustainable Dairy Research
- Skills I wanted to learn
- I wanted to better understand how software engineering can contribute to agricultural sustainability. My action plan was to learn about the research goals behind my tasks, participate in team discussions, and consider how data quality and reproducibility affect scientific work.
- Technologies and tools involved
- The project connected environmental measurements, animal information, automated data collection, and statistical analysis. My work focused on the software layer that helps researchers combine these sources and prepare them for further study.
- Why this mattered
- This mattered because the software was part of a broader effort to reduce the environmental impact of dairy production. Understanding that context helped me evaluate technical decisions by their effect on research quality rather than by implementation alone.
- Reflection
- This goal was partially completed. I learned that emissions comparisons are defensible only when researchers can establish which equipment produced a measurement and under what configuration. Building that provenance into the data model strengthened the evidence available for emissions research and showed me how a software decision can support a sustainability objective. The wider biological, economic, and producer-facing implications of sustainable dairy systems remain an area I want to study further.
Communicating: Written Communication
- Skills I wanted to learn
- I wanted to improve my ability to document technical work for people with different backgrounds. My action plan was to write concise progress updates, explain important assumptions alongside my code, and ask for feedback on whether my documentation was clear.
- Technologies and tools involved
- I practiced this skill through setup instructions, workflow documentation, code comments, test descriptions, and written project updates. My goal was to explain both what the software did and why particular decisions had been made.
- Why this mattered
- This mattered because research software is often maintained and used by people who did not originally write it. Clear documentation reduces confusion, makes results easier to reproduce, and helps future contributors change the system safely.
- Reflection
- This goal was partially completed. I produced setup guides, architecture documentation, data contracts, design decisions, and troubleshooting material for both developers and researchers. I improved substantially, but translating technical detail into concise language for every audience still requires deliberate practice.
Communicating: Oral Communication
- Skills I wanted to learn
- I wanted to become more confident explaining technical progress, design choices, and unresolved questions during meetings. My action plan was to prepare short summaries, ask focused questions when requirements were unclear, and connect my work back to the team’s research goals.
- Technologies and tools involved
- I discussed topics such as data organization, validation, automation, testing, and system reliability with both technical and research-focused team members. This required adjusting the level of detail to suit the audience.
- Why this mattered
- This mattered because many software decisions depended on scientific context that I could not determine from the code alone. Clear conversations helped the team identify incorrect assumptions early and make decisions with a shared understanding of the problem.
- Reflection
- This goal was completed. Meetings and presentations made me more confident explaining design choices, asking focused questions, and adapting the framing and level of detail for different audiences. These conversations also exposed assumptions that were not visible in the code or source files.
Additional takeaways
Reliable Pipelines Are Designed for Failure
Malformed files, duplicate inputs, interrupted batches, and reruns are normal operating conditions. Designing for them with validation, transactions, checksums, and locks made the pipeline safer to operate and easier to trust. Reliability was part of the feature, not cleanup after it worked.
A Data Model Is a Claim About the Real World
A schema can be internally consistent and still encode the wrong assumptions. Wherever software represents physical operations, identifiers, ownership, and time-dependent relationships should be checked against the real domain before they are made permanent in code.
The Same System Needs Different Explanations
Different audiences need different explanations. Technical audiences ask “How does it work?” and “Why was it built that way?” Non-technical audiences ask “What problem does it solve?”, “Why does it matter?”, and what they could do with it that they could not do before. Learning to switch between these perspectives made my presentations clearer and helped the team understand my work and its value.
Conclusion
My second co-op work term showed me how software engineering supports scientific work long before a final analysis is produced. I began with raw files from farm equipment and developed an installation-aware pipeline that validates, organizes, and stores gas measurements and milking sessions in a reproducible database.
The most important change was in how I approached engineering problems. Following the dependencies between equipment history, timestamps, validation, storage, and deployment taught me to stop treating each file as an isolated transformation and start designing the work as a system. I now judge a pipeline not only by whether it produces output, but by whether it preserves meaning, fails visibly, and can be understood and rerun by someone else.
The term also changed how I think about collaboration. Discussions with researchers supplied context that the code could not, while presenting and documenting the system for different audiences taught me to separate implementation detail from practical meaning. I am leaving the term more confident not only in building reliable software and data infrastructure, but in making it understandable to the people who will use and extend it.
Acknowledgements
I would like to thank the following members of CGIL for making this work term such a valuable experience:
- Dr. Christine Baes, for making this work term possible, supervising my work, and trusting me with the opportunity to contribute to the Net-Zero Dairy Genome Project.
- Dr. Ricarda Jahnel, for her guidance throughout the term and for helping me connect the software work to the research it was intended to support.
- Dr. Lucas Alcantara, for his technical and data-engineering mentorship, particularly around system architecture, data quality, and building reliable research workflows.
- Dr. Samla Cunha, for sharing her knowledge and helping me understand the research setting during the site visit.
- Dr. Dan Tulpan, for referring me to this position and recognizing that my software development experience would be a strong fit for the project.
- The graduate students at CGIL, for welcoming me into team meetings and engaging thoughtfully with my presentations. Their questions and perspectives helped me explain technical decisions more clearly and consider the work from perspectives beyond software engineering.