# Boston Police Field Interrogation and Observation (FIO) records, 2011 to 2025

## Facts

- Provider: City of Boston (Boston Police Department)
- License: Open Data Commons Public Domain Dedication and License (PDDL) 1.0
- Format: Parquet (in sampled files)
- Stored at: Amazon S3, s3://scigantic-boston-open-data/bpd-fio (us-east-1)
- Access: public, readable without an account
- Category: Urban & Civic
- Source: https://data.boston.gov/dataset/boston-police-department-fio
- Page: https://scigantic.com/archive/95278561-7c56-5dd5-8eff-ce7de4326691
- Keywords: Urban & Civic, time series, Parquet, AWS Open Data

## About

The Boston Police Department's record of Field Interrogation, Observation, Frisk and Search encounters, released by BPD for transparency and oversight of the FIO programme: 356,315 rows across 27 DataStore-loaded resources spanning 2011 to 2025, plus three field-description spreadsheets. BPD publishes it, in its own words, to add transparency to the FIO programme while protecting the privacy of the individuals involved, and it is the primary public record for research and oversight of stop, frisk and search practice in the city. The structure reflects three successive records management systems, and getting that structure right is the whole problem. The 2011 to 2015 legacy system is one flat 152,230-row file with 44 columns: a sequence and FIO id, the encounter date and time, the location and BPD district, whether the subject had priors, physical description, complexion, the FIO type (the FIOFS_TYPE code encodes which of interrogation, observation, frisk and search occurred), a terrorism flag, whether a search took place and its legal basis, the stop reason, the outcome, vehicle details, the officer and supervisor identifiers and names, and the subject's recorded sex, race and ethnicity. From 2015 onward the data is published as paired tables per year: a fieldcontact table with one row per encounter and a fieldcontact_name table with one row per person involved in an encounter, which is why the name-table row counts run roughly twice the contact-table counts. Joining them on the contact identifier is required to relate an encounter to the people in it, and counting rows in the name table counts people, not stops. Two data-quality facts are load-bearing and documented in the schema card. FIO_DATE in the legacy file ranges from 1964 to 2044, with 1,455 of 152,230 rows falling outside a plausible 2011 to 2016 window, so any time series has to filter to a sane date range first or the axis stretches over eighty years. And FIO_TIME is stored as a timestamp whose date component duplicates FIO_DATE rather than as a time of day, so hour-of-day analysis has to extract the time part rather than treating the column as a datetime. The schema card also documents the demographic column vocabularies (RACE_DESC, ETHNICITY, SEX) and the search-basis and outcome codes. Published by the Boston Police Department under PDDL 1.0. This archive is served from a Scigantic mirror rather than live from the portal: the yearly files are published once and not revised, so s3://scigantic-boston-open-data/bpd-fio/ carries a byte-verbatim copy under raw/ alongside a Snappy Parquet copy under parquet/, FUSE-mounted read-only at ~/archive.

## Load the data

```python
import os
import pandas as pd
df = pd.read_csv(os.path.join(os.environ['SCIGANTIC_MOUNT_PATH'], 'raw/rms_fieldcontacts_name_for_public_2017_202003111442.csv'), nrows=1000)
df.head()
```

Run in a Scigantic notebook on 2026-09-14 against `raw/rms_fieldcontacts_name_for_public_2017_202003111442.csv`: CSV/TSV: columns ['recnum', 'fc_num', 'contact_date', 'sex', 'race', 'age', 'build', 'hair_style', 'complexion', 'ethnicity', 'otherclothing'], dtypes ['int64', 'str', 'str', 'str', 'str', 'int64', 'str', 'str', 'str', 'str', 'str'], first 1000 rows read.

From any machine:

```python
import boto3
from botocore import UNSIGNED
from botocore.client import Config
s3 = boto3.client('s3', region_name='us-east-1', config=Config(signature_version=UNSIGNED))
s3.list_objects_v2(Bucket='scigantic-boston-open-data', Prefix='bpd-fio/', Delimiter='/')
```

Lists the files with anonymous access. Runs on any machine, with no cloud account.

## Data structure

File types in sampled files: Parquet

Example file: `bpd-fio/parquet/2024_annual_fio.parquet`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/95278561-7c56-5dd5-8eff-ce7de4326691
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/95278561-7c56-5dd5-8eff-ce7de4326691
- Over MCP: `get_schema_card` with id `95278561-7c56-5dd5-8eff-ce7de4326691` at https://api.scigantic.com/mcp

## Related topics

- [Parquet datasets](https://scigantic.com/topics/parquet)
- [Time series datasets](https://scigantic.com/topics/time-series)
- [Urban & Civic datasets](https://scigantic.com/topics/urban-and-civic)
