Blog

The latest Big Data in Agriculture news

Innovative surveillance technique gives vital time needed to track a cereal killer

By Hannah Craig / August 13, 2019

A new paper published in BMC Biology by the MARPLE diagnostics team, a 2018 Inspire Challenge Scale-up winner, shows how the research partnership reduced the speed of diagnostics from many months in high-end labs, to just two days from the side of an Ethiopian field.

Innovative surveillance technique gives vital time needed to track a cereal killer

Timor-Leste launches world-first monitoring system for small-scale fisheries

By Hannah Craig / August 2, 2019

A 2018 Inspire Challenge project led by WorldFish and Pelagic Data Systems has launched a new online dashboard that tracks fishing activities in Timor-Leste. The dashboard puts important data in the hands of fisheries officers, researchers and local stakeholders.

Timor-Leste launches world-first monitoring system for small-scale fisheries

Webinar summary – Can machine learning technologies be useful to create or complete ontologies in agriculture?

By Hannah Craig / June 25, 2019

The Ontologies Community of Practice launched a webinar series to debate, share, and advance our thinking on selected topics in the domain of ontologies. During this second webinar we learned about machine learning technologies and the link with ontologies.

Webinar summary – Can machine learning technologies be useful to create or complete ontologies in agriculture?

News

GARDIAN enables data discovery across CGIAR and beyond

By Hannah Craig / June 24, 2019

GARDIAN, the BIG DATA Platform's powerful search engine, helps CGIAR get its data together.

GARDIAN enables data discovery across CGIAR and beyond

GARDIAN: The data search tool unlocking agricultural development innovation

By Hannah Craig / June 18, 2019

Unlocking digital innovation in agricultural development requires the ability to aggregate data from a variety of sources and disciplines. GARDIAN enables users to do just that.

GARDIAN: The data search tool unlocking agricultural development innovation

[WEBINAR] Minimum Data Requirements for Crop Modeling

By Hannah Craig / June 18, 2019

CGIAR Webinar by the Platform for Big Data in Agriculture’s Community of Practice on Crop Modeling – Minimum Data Requirements for Crop Modeling

[WEBINAR] Minimum Data Requirements for Crop Modeling

CIAT, World Bank and partners announce Digital Agriculture Country Profiles initiative

By Hannah Craig / June 13, 2019

Building on the success of the Climate-Smart Agriculture Country Profiles, CIAT, together with the World Bank and FAO, is leading an initiative to create profiles for digital agriculture, starting with Argentina, Grenada, Kenya, Turkey and Vietnam.

CIAT, World Bank and partners announce Digital Agriculture Country Profiles initiative

Webinar – Can machine learning technologies be useful to create or complete ontologies in agriculture?

By Hannah Craig / June 12, 2019

CGIAR Webinar by the Platform for Big Data in Agriculture’s Community of Practice on Ontologies – Can machine learning technologies be useful to create or complete ontologies in agriculture?

Webinar – Can machine learning technologies be useful to create or complete ontologies in agriculture?

Subscribe for updates & other industry news

WATCH

June 18, 2019

Webinar

[WEBINAR] 2019 Inspire Challenge Q&A

[VIDEO] Navigating GARDIAN: A demonstration of the data search and discovery tool

June 18, 2019

Video

[VIDEO] Navigating GARDIAN: A demonstration of the data search and discovery tool

June 18, 2019

[WEBINAR] Minimum Data Requirements for Crop Modeling

June 12, 2019

Webinar – Can machine learning technologies be useful to create or complete ontologies in agriculture?

April 10, 2019

Webinar – Tale of Two Platforms: GEMS & GARDIAN

March 13, 2019

Webinar – Digital Extension in CGIAR

February 26, 2019

Webinar – What makes an ontology a reference for data annotation?

February 7, 2019

Issue Brief & Video – Overcoming Challenges to Digital Agribusiness Start-Ups in Developing Countries

New Applications for Data: CGIAR’s Promising New Research

January 28, 2019

Video

New Applications for Data: CGIAR’s Promising New Research

January 28, 2019

Challenges and opportunities of establishing data-driven agronomic services

January 22, 2019

Webinar – Origin of Asymmetry: The need for standards in data-driven services for agriculture

December 6, 2018

Webinar: Pier Luigi Buttigieg on publishing F.A.I.R data for agriculture development

YOUTH IN DATA

5 key challenges facing the agriculture data ecosystem

October 19, 2018

In adopting an emerging technology like Big Data, there are common issues that every industry must deal with to realize ...
Read More

3 sources of demand for big data in biodiversity innovation

October 17, 2018

Margaret Ann Tutwiler, the Director General of Bioversity International, discusses the importance of big data tools for supporting agricultural diversity initiatives ...
Read More

Convention highlights: Blockchain in food systems

October 15, 2018

Blockchain in Food Systems Blockchain is one of the most exciting emerging technologies and was given much attention during the ...
Read More

Kenya’s ICT Cabinet Secretary: Data is ‘key’ to smart farming

October 12, 2018

Kenya’s ICT Cabinet Secretary: Data is 'key' to smart farming Smart farming is key to developing sustainable agriculture and this ...
Read More

PRIVACY AND ETHICS

If Mary has a little lamb, who should know about it?

June 8, 2018

The GDPR and data science for agricultural development are at odds when it comes to delivering personalized products for the ...
Read More

Grow more food and mitigate climate change? Agricultural soil data could help

April 16, 2018

A more comprehensive understanding of agricultural soils is needed to support sustainable development decision making - but collecting the necessary ...
Read More

Meet Bob. The farmer needing your help to revolutionize agriculture

April 16, 2018

Farmers are contributing to the huge amounts of information that can be harnessed for global food security solutions. But they ...
Read More

Data democracy: Data from the people, by the people, for the people

March 16, 2018

If the farmer has right to consume, sell, or share their crop, then what about the data generated by the ...
Read More

INNOVATION

2019 Inspire Challenge Q&A webinar: Key takeaways

June 5, 2019

This year the CGIAR Platform for Big Data in Agriculture will award up to 1 million USD in grants to ...
Read More

When climate problems arise, farmers collect and share vital information with GeoFarmer app

April 17, 2019

The GeoFarmer app allows community workers and smallholder farmers to collect and share information on climate-related agricultural interventions. The innovative ...
Read More

Seeing really is believing: farmers’ photos revolutionize insurance and advisory services

January 25, 2019

When it comes to assessing insurance claims on damaged crops, seeing is believing. Which could be key to mitigating negative ...
Read More

Data-driven innovations taking agriculture to the digital frontier of science

November 12, 2018

At the intersection of agriculture and data science lies a host of exciting innovations that are pushing the boundaries to ...
Read More

ACCESS & DISCOVERABILITY

GARDIAN: The data search tool unlocking agricultural development innovation

June 18, 2019

Unlocking digital innovation in agricultural development requires the ability to aggregate data from a variety of sources and disciplines. GARDIAN ...
Read More

GARDIAN and facilitating data interoperability at CGIAR

February 12, 2019

The CGIAR online search engine, GARDIAN, is easy to navigate and to perform simple queries to locate data and publications; ...
Read More

Open Access: Unlocking agricultural innovation

October 23, 2018

Open Access: Unlocking agricultural innovation Open access to knowledge and data has the power to drive agricultural innovation, accelerating our ...
Read More

Grow more food and mitigate climate change? Agricultural soil data could help

April 16, 2018

A more comprehensive understanding of agricultural soils is needed to support sustainable development decision making - but collecting the necessary ...
Read More

APPLICATION & USE

Consuming information: Agriculture at the crossroads of sustainability

April 16, 2018

Extension officers, radio, and mobile phones remain crucial for the farmers to access and discover agricultural information ...
Read More

Transdisciplinary thinking essential to enable sustainable data-intensive agroecosystems

April 16, 2018

Transdisciplinary thinking is a fundamental requirement if we are to take meaningful steps towards using big data to solve complex ...
Read More

For the rural farmer, does big data raise more questions than answers?

April 12, 2018

Does big data raise more questions than answers? ...
Read More

Smallholder. Big data

April 12, 2018

Big data can help smallholders can become agriculture's big players ...
Read More

BIG DATA ACROSS CGIAR

Do you collect data from the field? We need to talk!

August 9, 2019

Georeferenced data has many potential uses for groundtruthing in data science applications, yet only about 20% of CGIAR-collected data is ...
Read More

A streamlined tool for rural household surveys – Q&A with Jim Hammond on RHoMIS – the Rural Household Multi-Indicator Survey

July 17, 2019

RHoMIS - the Rural Household Multi-Indicator Survey, aims to reduce the costs, time requirements and reporting burdens for those who ...
Read More

Mapping Suitability of Chicken Breeds in Ethiopia

July 9, 2019

Climatic suitability of animals has been less studied than crops, yet its potential has been increasingly recognized as critically important ...
Read More

Monitor, learn, evaluate: Tracking the impact of livestock development projects

July 9, 2019

Supporting Evidence Based Interventions (SEBI) recently gathered staff from eleven livestock development projects funded by the Bill and Melinda Gates ...
Read More

AgroFIMS: Your new companion for easy standardization of data collection and description

The Agronomy Field Information Management System (AgroFIMS) has been developed on CGIAR’s HiDAP (Highly-interactive Data Analysis Platform created by CGIAR’s International Potato Center, CIP). AgroFIMS draws fully on ontologies, particularly the Agronomy Ontology and the Crop Ontology. It consists of modules that represent the typical cycle of operations in agronomic trial management, and enables the creation of data collection sheets and digital data collection using the same ontology-based set of variables, terminology, units and protocols. AgroFIMS therefore:

Standardizes data collection and description for easy aggregation and inter-linking across disparate datasets;
Allows easy integration with HiDAP breeding data, or any other ontology-based datasets;
Functions as a data staging repository, allowing data uploads with view/edit permissions;
Enables data quality checks, statistical analysis of the data collected, and the generation of sophisticated statistics reports;
Aligns a priori with CGIAR’s CG Core metadata schema;
Enables easy upload to the institutional repositories, and much more.

Funding for AgroFIMS was provided by the Bill and Melinda Gates Foundation’s Open Access, Open Data Initiative, and the CGIAR Big Data Platform. Two user-testing workshops have been organized, with the most recent in March, with 14 participants including agronomists, crop modelers and developers from CGIAR centers (CIP, IITA, CIMMYT, Bioversity International, IFPRI), the University of Florida, and Rothamsted Research. A beta version will be released in summer for field testing and digital data collection using the KDSmart application developed in collaboration with Diversity Arrays Technology

Go to AGROFIMS →

Responsible Data Management Guidelines to protect privacy

CGIAR Platform for Big Data in Agriculture advocates open data for agricultural research for development. It considers that opening up research data for scrutiny and reuse confers significant benefits to society.

However, the Platform appreciates that not all research data can be open and that a broad range of legitimate circumstances may require data to be restricted.

As an integral component of its advocacy for open data, the Platform promotes responsible data management through the entire research data lifecycle from planning, collecting, storing, disclosing or publishing, transferring, discovery and archiving.

These guidelines were created from information collected from: review on best and emerging practices across various sectors in the fast changing landscape of privacy and ethics (130 external resources); privacy and ethic materials sourced from seven CGIAR centers; first draft was circulated for input and feedback across CGIAR and incorporated into this edition. It’s important to note that this is an evolving document, the next stage is to consult externally for further input.

These Guidelines are intended to assist agricultural researchers handle privacy and personally identifiable information (PII) in the research project data lifecycle.

Check the guidelines →

REUSE / TRANSFER

Ensure consistency with the DMP-PII and the purpose for which prior informed consent has been obtained
Revaluate likelihood of (re-)identification and risk of harm, particularly if it involves a public data-set containing PII (as above)
Ensure PII is stored securely to protect privacy (as above)
Minimize use of PII and risk of disclosure through pro-privacy access controls and analytical tools (as above)

Don’t transfer data containing PII unless have explicit consent
Don’t transfer data containing PII in the absence of a data sharing agreement identifying aspects such as purpose and scope of use, privacy protections measures, confidentiality and any limitations)
Don’t reuse or transfer PII until any inconsistencies with the DMP-PII and/or purpose compatibility have been resolved (e.g. through updated ethics review or consent from participant)

ARCHIVING / DISCARDING

Plan for archiving or data destruction early in the process. Destroying data can be more secure, however, archiving can be beneficial if the data has ongoing evidentiary, scientific or cultural value. If archiving, identify where and how, the budget require
Ensure DMP-PII and purpose compatibility (as above)
Ensure adequate security measures to protect privacy (as above)

Don’t wait until the end of the project to assess archiving needs when time and resources may be limited
Don’t assume the longevity of a particular format, future-proof your archives data
Don’t forget to budget for archiving data, this should be done as part of your Data Management Plan

PUBLISHING AND DISCOVERY

Ensure DMP-PII and purpose compatibility (as above)
Revaluate likelihood of (re-)identification and risk of harm, particularly if it involves a public data-set containing PII
Indicate in metadata the availability of raw data or minimized data containing PII, if available bilaterally
Minimize use of PII and risk of disclosure through pro-privacy access controls and analytical tools

Don’t include PII in public datasets unless absolutely necessary to preserve the data’s analytic potential, scientific utility or benefit to the participant (and subject to participants informed consent and a rigorous risk assessment)

STORAGE AND ANALYSIS

Ensure compatibility with the DMP-PII (as above) and also the purpose for which prior informed consent has been obtained

Ensure PII is stored securely to protect privacy, through organizational or project specific safeguards to prevent unauthorized access, accidental disclosure or breach of data (physical & technical)

encryption for the storage and transmission of PII
access control measures to limited access to PII
two-factor or multifactor authentication
cloud services & back-end security

Don’t store data in unsecured locations or on unsecured devices or servers

Don’t store encrypted data and encryption keys in locations where they can be easily accessed simultaneously

Don’t underestimate the importance and value of administrative safeguards to standardize practices (i.e. organizational policies, procedures and maintenance of security measures that are designed to protect private information, data and access)

COLLECTION

Ensure compatibility with the DMP-PII
De-identify data to anonymize by default unless it will impair the data’s analytic potential, scientific utility or benefit to the participant,
If you cannot anonymize, minimize the PII and pseudonymize to reduce the disclosure risk
Provide research participants sufficient information to use reasoned judgment to decide whether or not they wish to participate in the project
Ensure informed consent is designed to address the following elements:
- competence, comprehension, full disclosure, voluntariness
- legitimate scientific purpose for which the PII is collected and scope of use (e.g. stored, transferred, published and whether as anonymized, minimized or raw data)
- foreseeable risk of privacy loss and consequences
- meaningful alternatives including opt-in protection/anonymization
- safeguards to protect privacy, conditions on which PII may be shared and any limitations on reuse or third- party access and use of PII
- permission to follow-up or contact the participant and for what purpose (including by third- parties)
- participant’s right to withdraw and rights regarding their data (e.g. to be informed; to access; to rectify; to object; to erase)
- inclusion of physical, phone and/or electronic contact (at least two forms of contact) that participant can reach to exert her/rights
- explicit consent and participant’s acknowledgement of understanding
- if written, provide the participant a copy of processed informed consent
Use plain language and adapt informed consent to meet the needs of vulnerable populations (e.g. obtain orally or in local language)

Don’t collect PII unless you have a Data Management Plan and any necessary approvals in place, including the recorded approval of the potential participant
Don’t collect PII unless you absolutely need it
Don’t assume that removal of direct identifiers is sufficient to anonymize data or that all de-identification techniques will result in anonymized data. Consider the risk of re-identification of a research participant, particularly if datasets are combined. If there is a reasonable risk of re-identification the information should be handled as PII (i.e. undertake risk analysis, evaluate stronger anonymization techniques, seek informed consent for the disclosure of data and explain its possible consequences)
Don’t include vulnerable participants or communities if their ability or capacity to provide voluntary informed consent is genuinely in question
Don’t underestimate the potential of quasi or indirect identifiers to identify an individual, particularly the inherent ability of location-based data to identify participants and their communities, and the increased risk of harm this may pose to potentially vulnerable individuals/communities
Avoid seeking overly broad consent that may call into question transparency or a research participant’s understanding regarding the use of their PII, be specific regarding the activities, purpose and limitations associated with PII so that the participant can make a genuinely informed decision and downstream users can evaluate purpose compatibility and seek fresh consent if needed

PLANNING AND APPROVAL

Develop a Data Management Plan which governs the handling of PII in the research project and beyond (DMP-PII). It should address:
- the type and nature of PII
- compliance requirements (including necessary forms for obtaining consent, and ethics clearance, if applicable)
- legitimate research objectives that will be advanced by the PII
- foreseeable risks and consequences if participants are identified from the data
- privacy protection measures (or lack thereof) for collection, storage, transfer and publishing
- process for obtaining informed consent
- timeframe or trigger for archiving or deletion of PII
Employ stricter standards for research involving vulnerable populations such as children or illiterate participants or sensitive data such as ethnicity or religious beliefs
Undertake due-diligence of datasets previously collected by you or third parties to ensure you are entitled/permitted to use for your research project
Consult the legal, IRB or ethics clearance committee or any other relevant institutional group for specific institutional, local, regional or national policies and regulatory frameworks that may apply to PII in the context of your work

Don’t leave the handling of PII and privacy protection as an after-thought, plan ahead!
Don’t forget to check local laws and donor or third-party requirements in addition to institutional policies governing research ethics and privacy protection (seek expert support if unsure!)
Don’t ignore ethical practices/standards, if your institution does not have an ethics framework or clearance process in place self-assess!
In assessing whether information is capable of identifying someone (i.e. PII) don’t limit your focus to direct identifiers, also consider indirect/quasi identifiers. Appreciate this will depend on the context of the research project, the data in question and external data which is or may become otherwise available (i.e. there is no exhaustive list).
In assessing risk of harm don’t forget to consider potential harm to the participant’s community or groups of individuals that can otherwise be identified or associated with the participant

Blog

Subscribe for updates & other industry news

Success!

WATCH

YOUTH IN DATA

PRIVACY AND ETHICS

INNOVATION

ACCESS & DISCOVERABILITY

APPLICATION & USE

BIG DATA ACROSS CGIAR

AgroFIMS: Your new companion for easy standardization of data collection and description

Responsible Data Management Guidelines to protect privacy

REUSE / TRANSFER

ARCHIVING / DISCARDING

PUBLISHING AND DISCOVERY

STORAGE AND ANALYSIS

COLLECTION

PLANNING AND APPROVAL