Methodology

Methodology

Mon Feb 19 2018
An investigation about attacking Syrian hospitals

This report took an interdisciplinary approach towards investigating attacks on medical facilities in Idlib during January and February 2018. In the report, the authors have included a variety of sources for analysis and investigation, which each have their own respective methodologies. Specific methodologies are provided in the following pages:

Witness Statements

Following the hospital attacks in Idlib province during the month of January and February 2018, Syrians for Truth and Justice established a field research team which was tasked with entering the city and inspecting the impact sites. These organisations were additionally tasked with collecting material evidence as well as accounts of the survivors, such as those injured and their families, as well as accounts of eyewitnesses (e.g. medical staff; managers of the hospitals; civil defense team members).

Interviews were conducted in person in Idlib by staff members of their respective organisations and recorded on audio devices and then later transcribed. Staff members conducted a total of 10 semi-structured interviews using a standardised questionnaire. Questions asked to respondents focused on the following themes:

  1. Details surrounding the attack (e.g. date, time, location)

  2. The number of patients each medical facility provided care to on a monthly basis

  3. The types of departments or procedures medical facilities conducted

  4. The geographic areas served by medical facilities

  5. Whether this medical facility had been previously subject to attack

  6. Information regarding casualties and those injured as a result of the attack

Flight Observation Data

To cross-reference with findings from visual content, flight observation data information was provided to the Syrian Archive by an organisation employing a well-developed network of spotters (flight observers) of aircrafts departing from military air fields primarily located in northwestern and central Syria.

Flight data and the visual content were analysed to identify whether flights were observed in the vicinity of locations attacked for locations in which aerial bombings were alleged.

Visual Content

The Syrian Archive employed its Digital Evidence Workflow, based off of the Electronic Discovery Reference Model developed by Duke University School of Law. This workflow consists of five components: A) Identification; B) Collection and secure preservation; C) Processing, verification and analysis; D) Review; and E) Publication. A "Do No Harm" ethical framework has been applied to all steps in the digital evidence workflow. Detailed methodologies of these components are analysed in the following subsections.

workflow

A. Identification

The Syrian Archive's identification process has three steps: 1) Establish a database of credible sources for digital content; 2) Establish a database of credible sources for verification; 3) Establish a standardised metadata schema. These three processes are outlined in detail below:

1) Establish database of credible sources for content

Before any collection, archival, or verification of digital materials was possible, the Syrian Archive first established a database of credible sources for visual content. The Syrian Archive project worked to identify over 3000 credible sources, a list consisting of individual journalists and field reporters, larger media houses (e.g. local and international news agencies), human rights organisations (e.g. Syria Institute for Justice), Syrian Civil Defense (White Helmets), and local field clinics and hospitals, and others. Many of the sources used by the Syrian Archive began publishing or providing visual content in late 2011-early 2012 and have also published work in other credible media outlets.

Credibility was determined by analysing whether the source is familiar to the Syrian Archive or to its existing professional network of Syrian journalists, media activists, human rights groups and humanitarian workers; whether the source's content and reportage been reliable in the past. This is determined by evaluating how long the source has been reporting and how active they are. To determine where the source is based, social media channels are evaluated to determine whether videos uploaded are consistent and mostly from a specific location where the source is based, or if locations differ significantly. Channels are analysed to determine whether the video account uses a logo and whether this logo is consistently used across videos. Channels are additionally analysed for original content to determine whether the uploader aggregates videos from other news organisations and YouTube accounts or whether they upload mostly user-generated content.

2) Establish database of credible sources for verification

Secondly, the Syrian Archive project worked to establish a database of credible sources for verification. These sources provide additional information used for verification of content originating on social media platforms or sent from sources directly. Those verifying content are made up of citizen journalists, human rights defenders and humanitarian workers based in Syria and abroad. To preserve data integrity, sources used for content did not comprise part of the database for verification.

3) Establish standardised metadata scheme

Third, the Syrian Archive recognised the need for a standardised metadata scheme for organising content, but also that any metadata scheme used would be a highly political choice. Given that there are no universally accepted legally admissible metadata standards as of the date of this publication, efforts were made to develop a framework in consultation with a variety of international investigative bodies. Among these include consultations with members of the International Criminal Court, with members of the United Nations Office for High Commissioner of Human Rights, with members of the International, Impartial and Independent Mechanism on international crimes committed in Syria (IIIM), with archival institutes like the NIOD Institute for War, Holocaust and Genocide Studies, with international human rights organisations like Amnesty International, Human Rights Watch, and Witness, and with research institutes like the Human Rights Center at UC Berkeley School of Law.

Establishing a standardised metadata schema is necessary in order to assist users in identifying and understanding when, where, and what happened in a specific incident. A review of practices by other war archival institutes, such as those of NIOD, found that additional information is helpful for contextualising raw visual content (e.g. location of video recording; date of video recording and upload; and the origin of the video). Metadata collected by the Syrian Archive project includes description of the visual object as given (e.g. YouTube title); the source of the visual content; the original link where footage was first published; specific landmarks able to be identified; weather (which may be useful for geolocation or time identification); specific languages or regional dialects spoken; clothes or uniforms able to be identified; weapons or munitions used; device used to record the footage; and media content type. The metadata is populated automatically and manually depending on how it was collected from e.g open source or closed source. A detailed description and full list of metadata field types are provided on the Syrian Archive website.

In categorising violations, the Syrian Archive has decided to use the violations categories used by the Office of United Nations High Commissioner for Human Rights (OHCHR). This was done because OHCHR is one of the groups in the unique position of being able to investigate incidents of human rights violations and war crimes. These categories consist of many often overlapping categories. Categories identified by the UN OHCHR Inquiry on Syria and used by the Syrian Archive project include:

Violations: treatment of civilians & hors de combat fighters

  • Massacres and other unlawful killing;
  • Arbitrary arrest and unlawful detention;
  • Hostage-taking;
  • Enforced disappearance;
  • Torture and ill-treatment of detainees;
  • Sexual and gender-based violence;
  • Violations of children's rights;

Violations: conduct of hostilities

  • Unlawful attacks;
  • Violations against specifically protected persons and objects;
  • Use of illegal weapons;
  • Sieges and violations of economic, social and cultural rights;
  • Arbitrary and forcible displacement.

Should potential investigations by international bodies not be pursued by the UN OHCHR and rather by another investigative body, it is anticipated that the Syrian Archive will modify violations categories to meet the needs of those investigating.

B. Collection and secure preservation

The collection and secure preservation of the digital evidence workflow ensures that the original content is not lost due to removal on corporate platforms. This is done by collecting and securely storing digital content on external backend servers before it goes through basic verification. It is then backed up securely on servers throughout the world. Videos are hashed with (SHA-256) and (Md5) consistent with current best practices and timestamped to ensure they are not tampered with after being collected from social media platforms (open source) or taken directly from sources (closed source). Simultaneously it is hashed and timestamped by an independent and impartial third party for reference and integrity purposes. Once verified, content is centrally published on the Syrian Archive website in an open-source format. The Syrian Archive uses the SugarCube software for this process, a free and open source software developed for use in human rights investigations using online-based user generated content research.

C. Processing, verification, and analysis

After content has been collected and stored securely, the next stage of the digital evidence workflow refers to the processing, verification, and analysis of digital materials. Detailed descriptions of these three components of the digital evidence workflow are outlined below:

1) Processing

Metadata from visual content collected from social media platforms is parsed and aggregated automatically using a predefined and standardised metadata scheme, as described above. Metadata from visual content sent to the Syrian Archive directory is created manually using the standardised metadata scheme.

This prepares the visual content for initial verification. As much additional metadata and chain of custody information as possible is recorded. This is done to assist users in identifying and understanding when, where, and what happened in a specific incident.

2) Verification

Verification is comprises of three components: 1) Verify the source of the video uploader; 2) Verify the location where the video was filmed; 3) Verify the dates in which the video was filmed and uploaded. Detailed descriptions of these three processes are outlined below:

  • Verify the source of the video uploader

Establish that the source of the video on the Syrian Archive's verified list of credible sources. If the source is not an existing trusted source, determine the new source's credibility by going through the procedure highlighted above.

In some cases, near-duplicate content may be published. For example, if one video is 30 seconds and a second video is 10 minutes but includes all or portions of the first video, both videos would be published as long as it is possible to verify both videos. Similarly videos from news organisations or media houses featuring all or parts of content from other videos are also preserved, as long verification is possible. The Syrian Archive also preserves duplications if they are from different sources and the original uploader is unable to be determined (for example if two identical videos are uploaded simultaneously).

The video uploader source may not necessarily be the same as the source who originally filmed content. In most of the video footage verified by the Syrian Archive, only the video uploader and not the video filmer is known. Advanced verification in the analysis phase includes the source of filming, a process done in cases deemed priority.

  • Verify the location where the video was filmed

Each video goes through basic geolocation to verify that it has been captured in Syria. More in-depth geolocation is conducted for priority visual content in order to verify that it has been captured in a specific location. This has been done by comparing reference points (e.g. buildings, mountains ranges, trees, minarets) with Google Earth satellite imagery, Microsoft Bing, and Digital Globe, as well as OpenStreetMap imagery and geolocated photographs from Google Maps. In addition to this, the Syrian Archive has referenced the Arabic spoken in videos against known regional accents and dialects within Syria to further verify location of videos. When possible, the Syrian Archive has contacted the source directly in order to confirm the location, and cross-reference video content by consulting existing networks of journalists operating inside and outside Syria to confirm the locations of specific incidents.

  • Verify the dates in which the video was filmed and uploaded

The Syrian Archive verifies the date of capturing the video by cross referencing the publishing date of visual content collected from social media platforms (e.g YouTube, Twitter, Facebook and Telegram) with dates from reports concerning the same incident. Visual content collected directly from sources is also cross referenced with reports concerning the same incident featured in the video.

  • News reports from international and local media outlets, including Reuters, Smart News Agency, Aleppo Media Center, Qasioun News Agency, LCC;
  • Human rights reports published by international and local organisations, including Human Rights Watch, Amnesty International, Syrian Human Rights Network, Violations Documentation Center in Syria, Syrian American Medical Society, and Physicians for Human Rights;
  • Reports shared by the Syrian Archive's network of citizen reporters on Twitter, Facebook and Telegram about the incidents.

Additional tools are used to check the date of the visual content such as Google reverse imagery and Sun Calc.

3) Analysis

In some cases, the Syrian Archive is able to conduct in-depth open source investigations. Time and capacity limitations means not all incidents are able to be analysed in-depth, however by developing a replicable workflow it is hoped that others can assist in these efforts of investigate other incidents using similar methods. A detailed overview of in-depth incident analysis is provided in the investigations section of the Syrian Archive website.

D) Review

Once digital materials have been processed, verified, and analysed, it is then reviewed for accuracy. In the event of a discrepancy, content is fed back into the digital evidence workflow for further verification. If content is deemed accurate it moves to the publishing stage of the digital evidence workflow.

E) Publication

Once the visual content is verified and reviewed, it's then published on the Syrian Archive database where they are made publicly available in a free and open source format. Regular reports on verified visual content ensure that the feedback loop between the Syrian Archive and sources who filmed the videos is closed. This allows the Syrian Archive to add value to the visual content being preserved, verified and analysed immediately for advocacy purposes and later on for accountability and justice purposes.