juliet

This guide was co-written by Andrew Zuker with support from the Heinrich Boell Foundation.

The U.S. government publishes volumes of detailed data on the money it spends, but searching through it and finding information can be challenging. Complex search functions and poor user interfaces on government reporting sites can hamper an investigation, as can inconsistent company profiles and complex corporate ownership structures. 

This week, EFF and the Heinrich Boell Foundation released an update to our database of vendors providing technology to components of the U.S. Department of Homeland Security (DHS), such as Immigration and Customs Enforcement (ICE) and Customs and Border Protections (CBP). It includes new vendor profiles, new fields, and updated data on top contractors, so that journalists and researchers have a jumping-off point for their own investigations.

Access the dataset through Google Sheets (Google's Privacy Policy applies) or download the Excel file here. 

This time we thought we would also share some of the research methods we developed while assembling this dataset.

This guide covers the key databases that store information on federal spending and contracts (often referred to as "awards"), government solicitations for products and services, and the government's "online shopping superstore," plus a few other deep-in-the-weeds datasets buried in the online bureaucracy. We have provided a step-by-step guide for searching these sites efficiently and help tips for finding information. While we have written this specifically with DHS agencies in mind, it should serve as a useful resource for procurement across the federal government. 

1. Procurement Sites: FPDS.gov and USASpending.Com 

Federal Procurement Data System - fpds.gov

The Federal Procurement Data System (FPDS) is the best place to start for finding out what companies are working with DHS. It is the official system for tracking federal discretionary spending and contains current data on contracts with non-governmental entities like corporations and private businesses. Award data is up-to-date and includes detailed information on vendors and awards which can be helpful when searching the other systems. It is a little bit old-school, but that often makes it one of the easiest and quickest sites to search, once you get the hang of it, since it offers a lot of options for narrowing search parameters to specific agencies, vendors, classification of services, etc. 

How to Use FDPS
To begin searching Awards for a particular vendor, click into the “ezSearch” field in the center of the page, delete or replace the text “Google-like search to help you find federal contracts…” with a vendor name or keywords, and hit Enter to begin a new search. 

A new tab will open automatically with exact matches at the top. 

Four “Top 10” modules on the left side of the page link to top results in descending order: Department Full Name, Contracting Agency Name, Full Legal Business Name, and Treasury Account Symbol. These ranked lists help the user quickly narrow in on departments and agencies that vendors do business with. DHS may not appear in the “Top 10” results, which may indicate that the vendor hasn’t yet been awarded DHS or subagency contracts.

For example, if you searched the term “FLIR”, as in Teledyne FLIR who make infrared surveillance systems used along the U.S.-Mexico border, DHS is the 2nd result in the “Top 10: Department Full Name” box. 

To see all DHS contracts awarded to the vendor, click “Homeland Security, Department of” from the “Top 10 Department Full Name” module. When the page loads, you will see the subcomponents of DHS (e.g., ICE, CBP, or the U.S. Secret Service) in the lefthand menu. You can click on each of those to drill down even further. You can also drill down by choosing a company. 

Sorting options can be found on the right side of the page which offer the ability to refine and organize search results. One of the most useful is "Date Signed," which will arrange the results in chronological order. 

You don't have to search by a company name. You can also use a product keyword, such as "LPR" (license plate reader). However, because keywords are not consistently used by government agencies, you will need to try various permutations to gather the most data. 

Each click or search filter adds a new term to the search both in the main field at the top and in the Search Criteria module on the right. They can be deleted by clicking the X next to the term in this module or by removing the text in the main search field.

For each contract item, you can click "View" to see the specific details. However, these pages don't have permalinks, so you'll want to print-to-pdf if you need to retain a permanent copy of the record. 

Often the vendor brand name we know from their marketing or news media is not the same entity that is awarded government contracts. Foreign companies in particular rely on partnerships with domestic entities that are established federal contractors. If you can’t find any spending records for a vendor, search the web for information on the company including acquisitions, partnerships, licensing agreements, parent companies, and subsidiaries. It is likely that one of these types of related companies is the contract holder. 

USA Spending - usaspending.gov

The Federal Funding and Accountability Act (FFATA) of 2006 and the DATA Act of 2014 require the government to publish all spending records and contracts on a single, searchable public website, including agency-specific contracts, using unified reporting standards to ensure consistent, reliable, searchable data. This led to the creation of USA Spending (usaspending.gov). 

USA Spending is populated with data from multiple sources including the Federal Procurement Data System (fpds.gov) and the System for Awards Management (sam.gov - which we'll discuss in the next section). It also compiles Treasury Reports and data from the financial systems of dozens of federal agencies. We relied heavily on Awards data from these systems to verify vendor information including contracts with the DHS and its subagencies such as CBP and ICE. 

USA Spending has a more modern interface, but is often very slow with the information often hidden in expandable menus. In many ways it is duplicative of FPDS, but with more features, including the ability to bookmark individual pages. We often found ourselves using FPDS to quickly identify data, and then using the "Award ID" number to find the specific record within USA Spending. 

USA Spending also has some visualizations and ways to analyze data in chart form, which is not possible with the largely text-based FPDS. 

How to Use USA Spending

To begin searching for DHS awards, click on either “Search Award Data” on the navigation bar, or the blue “Start Searching Awards”button. 

On the left of the Search page are a list of drop down menus with options. You can enter a vendor name as a keyword, or expand the “Recipient” menu if you know the full company name or their Unique Entity Identifier (UEI) number. Expand the “Agency Tab” and enter DHS which will bring up the Department of Homeland Security Option.

In the example below, we entered “Palantir Technologies” as a keyword, and selected DHS in the Agency dropdown:

For vendors with hundreds of contracts that return many pages of results, consider adding more filters to the search such as a specific time period or specifying a Funding Agency such as ICE or CBP. In this example, the filters “Palantir Technologies” and “DHS” returned 13 results (at the time of publication). It is important to note that the search results table is larger than what displays in that module. You can scroll down to view more Awards and scroll to the right to see much more information. 

Scroll down outside of that module to reveal more info including modules for Results by Category, Results over Time, and Results by Geography, all of which can be viewed as a list or graph. 

Once you've identified a contract, you can click the "Prime Award ID" to see the granular details for each time. 

From the search, you can also select just the agency to see all the contracts on file. Each agency also has its own page showing a breakdown for every fiscal year of how much money they had to spend and which components spent the most. For example, here's DHS's page.

2. Contracting Opportunities  - SAM.gov  

So far we've talked about how to track contracts and spending, but now let's take a step back and look at how those contracts come to be. The System for Award Management, SAM.gov, is the site that allows companies to see what products and services the government intends to buy so they can bid on the contract. But SAM.gov is also open to the public, which means you can see the same information, including a detailed scope of a project and sometimes even technical details. 

How to Use Sam.gov

SAM.gov does not require an account for its basic contracting opportunity searches, but you may want to create one in order to save the things you find and to receive keyword- or agency-based alerts via email when new items of interest are posted. 

First you will click "Search" in the menu bar, which will bring you to this page: 

We recommend selecting both "Active" and "Inactive" in the Status menu. Contracts quickly go inactive, and besides, sometimes the contracts you are most interested in are several years old. 

If you are researching a particular technology such as unmanned aerial vehicles, you might just type "unmanned" in the Simple Search bar. That will bring up every solicitation with that keyword across the federal government.

One of the most useful features is filtering by agency, while leaving the keyword search blank. This will return a running list of an agency's calls for bids and related procurement activities. It is worth checking regularly. For example, here's what CBP's looks like on a given day: 

If you click on an item, you should next scroll down to see if there are attachments. These tend to contain the most details. Specifically, you should look for the term "SOW," the abbreviation for "Statement of Work." For example, here are the attachments for a CBP contracting opportunity for "Cellular Covert Cameras": 

The first document is the Statement of Work, which tells you the exact brand, model, and number of devices they want to acquire: 

The attachments also included a "BNO Justification." BNO stands for "Brand Name Only," and this document explains in even more detail why CBP wants that specific product:

If you see the terms "Sole Source" in a listing, that also means that an agency has decided that only one product meets its requirements and it will not open bidding to other companies. 

In addition to contracting, many agencies announce "Industry Day" events, usually virtual, that members of the public can join. This is a unique opportunity to listen in on what contractors are being told by government purchasing officials. The presentation slides are also often later uploaded to the SAM.gov page. Occasionally, the list of attendees will also be posted, and you'll find several examples of those lists in our dataset.

3. The Government's "Superstore" - gsaadvantage.gov

Another way to investigate DHS purchasing is by browsing the catalog of items and services immediately available to them. The General Services Administration operates GSA Advantage, which it describes as "the government's central online shopping superstore." The website's search is open, allowing members of the public to view any vendors' offerings–including both products and services– easily as they would with any online marketplace. 

For example, you could search for "license plate reader" and produce a list of available products: 

If you click "Advanced Search," you can also isolate every product available from a particular manufacturer. For example, here are the results when you search for products available from Skydio, a drone manufacturer.

If you switch from "Products" to "Services" you can export datasets for each company about their offerings. For example, if you search for "Palantir" you'll get results that look like this:

This means all these companies are offering some sort of Palantir-related services. If you click "Matches found in Terms and Conditions," you'll download a PDF with a lot of details about what the company offers. 

For example, here's a a screengrab from Anduril's documentation: 

If you click "Matches Found in Price List" you'll download a spreadsheet that serves as a blueprint of what the company offers, including contract personnel. Here's a snippet from Palantir's: 

4. Other Resources

Daily Public Report of Covered Contract Awards - Maybe FPDS isn't enough for you and you want to know every day what contracts have been signed. Buried in the DHS website are links to a daily feed of all contracts worth $4 million or more. It's available in XML, JSON, CSV and XLSX formats. 

DHS Acquisition Planning Forecast System (APFS) - DHS operates a site for vendors to learn about upcoming contracts greater than $350,000. You can sort by agency at a granular level,  such as upcoming projects by ICE Enforcement & Removal Operations. This is one to check regularly for updates. 

DHS Artificial Intelligence Use Case Inventory - Many federal agencies are required to maintain datasets of "AI Use Cases." DHS has broken these out for each of its subcomponents, including ICE and CBP. Advanced users will find the spreadsheet versions of these inventory more interesting. 

NASA Solutions for Enterprise-Wide Procurement (SEWP) - SEWP is a way for agencies to fast track acquisition of "Information Technology, Communication and Audio Visual" products through existing contracts. The site provides an index of existing contract holders, but the somewhat buried "Provider Lookup" has a more comprehensive list of companies involved in this type of contracting, illustrating how the companies serve as passthroughs for one another. Relatedly, DHS's list of "Prime Contractors" shows which companies hold master contracts with the agency and its components. 

TechInquiry - Techinquiry is a small non-profit that aggregates records from a wide variety of sources about tech companies, particularly those involved in government contracting.