Skip to content
Home Posts About us Privacy Policy
Sharing Social Media Datasets: Identifiers, Aggregates, Controlled Access or Open Release

Sharing Social Media Datasets: Identifiers, Aggregates, Controlled Access or Open Release

Share ๐• in f W

Once a project has collected public social media data, someone will ask for it: a journal reviewer, a co-author at another institution, a client, or a team that wants to repeat the analysis. The decision is how to share it, and there are four realistic options: share only the identifiers of the posts, share aggregated or derived data, share full records under controlled access, or publish the full records openly. For most projects the first two cover nearly every legitimate request, the third fits when others need the raw text, and the fourth is rarely compatible with platform terms or with the privacy of the people in the data. The sections below compare each option against the same criteria, so the trade-offs are visible side by side.

The criteria that decide the choice

Six questions separate the options:

  • Platform terms: what the developer agreement or terms of service allow you to redistribute, and to whom.
  • Privacy: how much personal data leaves your control, and whether people can still be identified.
  • Reproducibility: whether someone else can rebuild your results from what you share.
  • Deletion: whether a post that its author deletes disappears from the shared copy too.
  • Effort: the work for you and for the recipient.
  • Longevity: whether the shared material will still be usable in five years.

Platform terms come first because they can rule an option out entirely. They also change, so the version in force when you share matters more than the one in force when you collected. Record the date and the text you relied on.

Option 1: Sharing post identifiers only

The shared dataset is a list of post or account identifiers, sometimes with the date of collection, and nothing else. A recipient who wants the content requests it again from the platform, usually through the official API, a process often called rehydration.

  • Platform terms: some platforms have allowed identifiers to be shared where full content could not be. Whether that applies today depends on the current terms, so check before relying on it.
  • Privacy: strong. The identifiers alone carry little, and the recipient only gets what the platform still serves under its own rules.
  • Reproducibility: partial, and it decays. Posts deleted or made private since collection cannot be retrieved, so the recipient works with a smaller dataset.
  • Deletion: handled automatically, which is the main reason this approach exists.
  • Effort: low for you, higher for the recipient, who needs API access that may cost money or require approval.
  • Longevity: weak. If the platform changes its API or closes access, the list cannot be turned back into data.

Suppose, hypothetically, a study is based on 40,000 posts and a reviewer rehydrates the list a year later, recovering 34,000. The reviewer can check whether the main findings hold on the remaining posts, but not reproduce every number exactly. Stating that limitation in the paper, together with the collection dates, prevents a mismatch from looking like an error.

An identifier list is more useful when it carries a little structure. Alongside each identifier, include the collection date, the query or account list that produced it, and a flag showing which analyses used it. A recipient can then rebuild one figure without rehydrating the whole set, and can see whether the posts that disappeared were concentrated in one part of the sample, which matters for interpreting the gap.

Option 2: Sharing aggregated or derived data

Instead of posts, you share what you computed from them: counts per day, share of posts per topic, network statistics, model outputs per group, or coded categories without the underlying text.

  • Platform terms: usually the most permissive case, because derived statistics are not the platform's content. Some terms still restrict certain derived uses, such as information about individual users, so check.
  • Privacy: strong when groups are large. Small groups can still reveal individuals; a count of one is a person.
  • Reproducibility: good for the analysis that follows aggregation, none for the steps before it. A recipient can check your regression, not your coding.
  • Deletion: mostly not an issue, since individual posts are not present, although figures computed partly from posts later deleted remain in the totals.
  • Effort: low for both sides.
  • Longevity: excellent. A file of counts does not depend on any API.

The common mistake here is sharing aggregates with too fine a breakdown. A table of posts per account per day is effectively a record of individual behaviour. A simple rule is to set a minimum cell size, for example suppressing any cell based on fewer than ten accounts in a hypothetical project, and to apply it before export rather than trusting the recipient to ignore small cells.

Derived data also includes coded labels and model outputs. A file that lists, for each post identifier, the topic or sentiment your team assigned sits between Options 1 and 2: it lets others check your coding against the rehydrated text, but it also attaches an interpretation to identifiable posts. That is usually acceptable for neutral categories such as topic, and needs more thought when the label touches a sensitive trait, such as inferred political leaning or health.

Option 3: Sharing full records under controlled access

The full dataset, including post text and metadata, is made available to specific people under a written agreement. This can be a direct data sharing agreement between two institutions, a restricted-access deposit in a research data repository, or access inside a secure environment where the recipient can analyse the data but not download it.

  • Platform terms: the hardest question. Some terms allow sharing with collaborators for the same project or with approved researchers; others forbid any redistribution of content. Read the clauses on redistribution and on third parties carefully.
  • Privacy: moderate. Personal data leaves your hands, so the agreement must limit use, require security, forbid re-identification and set a deletion date. In the EU and the UK, the recipient may become a separate controller, which brings its own obligations.
  • Reproducibility: the best of all options, since the recipient sees exactly what you analysed.
  • Deletion: must be handled by agreement, for example a commitment to apply deletions you pass on, or a fixed expiry.
  • Effort: high. Agreements take time, repositories have review processes, and secure environments cost money to run.
  • Longevity: good while the repository or agreement lasts.

Controlled access is the right answer when the research question depends on the text itself, for example in qualitative coding that another team needs to verify. Ethics committees and data protection officers often have templates for these agreements, which saves drafting time.

Consider a hypothetical case: two universities study public discussion of a local election, and the second team wants to recode a sample of 2,000 posts with a different scheme. A short agreement limiting use to that recoding, requiring storage on institutional systems, forbidding contact with the authors of posts and setting deletion at the end of the project gives the second team what it needs without creating a copy that outlives the purpose. Both sides keep a signed copy with the dataset documentation.

Option 4: Publishing full records openly

The full dataset is posted on a public website or open repository for anyone to download.

  • Platform terms: often prohibited, especially for post text and user information.
  • Privacy: weak. Once public, the data cannot be withdrawn, and people who posted for a small audience may find their words in contexts they never expected.
  • Reproducibility: excellent in principle.
  • Deletion: not possible in practice; copies spread.
  • Effort: low at first, potentially high later when removal requests arrive.
  • Longevity: excellent, which is part of the problem.

There are narrow cases where open publication can fit, such as posts from official institutional accounts on a platform whose terms allow it. Even then, replies and mentions from private individuals usually come along with the institutional posts, and they need the same care as any other personal data. If in doubt, choose Option 3.

The four options side by side

CriterionIdentifiers onlyAggregatesControlled accessOpen publication
Platform termsOften allowed, checkUsually allowedDepends on clausesOften prohibited
PrivacyStrongStrong with minimum cellsModerate, by agreementWeak
ReproducibilityPartial, decaysDownstream onlyFullFull
DeletionAutomaticMostly not an issueBy agreementNot possible
EffortLow for youLowHighLow, then high
LongevityWeakExcellentGoodExcellent

Combining options in one release

Most strong data releases combine two options. A frequent pattern is identifiers plus aggregates: the identifiers let others rebuild what remains of the raw data, and the aggregates preserve the exact numbers behind the published figures. Another is aggregates for everyone plus controlled access to full records for verified researchers.

Whichever combination you choose, include a short data statement: which platform and access route, the collection dates, the query or sampling method, what was removed and why, and the terms under which the release is offered. A recipient who knows how the data was gathered can judge whether it suits a new question. The practices described in ethical and effective social media scraping best practices support writing that statement, because a well-documented collection is easier to describe.

If collection was handled by a provider, such as ScrapingIsNotACrime or another service, check what your agreement with that provider says about onward sharing. The provider's terms and the platform's terms can both apply.

Mistakes that make sharing go wrong

  • Deciding at publication time. Consent forms, ethics approvals and API applications often describe sharing. Decide early so they say the right thing.
  • Removing usernames and calling the text anonymous. A searchable quote leads straight back to its author.
  • Forgetting the derived files. Model training sets, coded spreadsheets and intermediate exports can contain full text even when the main release does not.
  • Mixing options without saying so. A release that contains identifiers for most posts and full text for a few quoted ones should state which is which, so recipients do not assume every record can be redistributed the same way.
  • No contact point. People who find their posts referenced in a release need someone to write to. A named role or shared address, kept active for the life of the release, is enough.
  • Not passing on deletions. When a controlled-access recipient holds a copy, a process for sending them deletion lists keeps both copies aligned.

Testing a release before it goes out

A short test catches most problems before they become public. Give the release, exactly as it will be shared, to a colleague who did not work on the project. Ask them to try to find one specific person from it within half an hour, using only ordinary web searches. If they succeed through a quote, a rare combination of location and date, or a small cell in a table, the release needs another pass: paraphrase or drop the quote, coarsen the dates, or merge small groups.

Then check the release against the platform terms one last time, clause by clause, and record the result with the date. This test does not prove a release is safe, but it finds the obvious failures, and the record shows that the question was asked.

Which option fits which reader

For an academic researcher publishing a quantitative study, identifiers plus aggregates at a safe level of detail usually satisfy journals and keep the release within common platform terms. Add controlled access if reviewers or replication teams need the text.

For a qualitative researcher whose findings rest on close reading, controlled access is the realistic route, supported by an agreement that sets purpose, security and an end date.

For a company sharing results with a client or partner, aggregates are the default. Raw posts should move only when a contract covers it and the platform terms allow it, and a lawyer should review that contract where personal data is involved.

For a journalist or public-interest team, aggregates and carefully chosen quotes from public figures or official accounts usually tell the story without releasing data about private individuals. The seven questions to answer before collecting public social media data for research cover the collection choices that make a later release easier.

Share ๐• in f โœˆ