Every social media dataset faces the same decision sooner or later: what happens to a post after its author deletes it, edits it or makes the account private. For most projects, the sound answer is that removed content leaves the working dataset on a fixed schedule, set by the access terms and the project's data plan, and that edits replace earlier text unless there is a documented reason to keep versions. This guide covers the kinds of change to expect, where the obligations come from, how to detect changes, what to do with each one, and how the choice differs for researchers, analysts and product teams.
What can change after a post is collected
A collected post is a snapshot of something its author still controls. Five kinds of change are common, and they do not mean the same thing:
| Change | What it usually signals | Typical handling |
|---|---|---|
| Post deleted by its author | The author no longer wants it public | Remove the content |
| Post edited | The author corrected or changed it | Replace the text with the current version |
| Account made private or protected | The author restricted the audience | Treat the content as no longer public |
| Account deleted or deactivated | The author left the platform | Remove content and profile fields |
| Post or account removed by the platform | A moderation or legal action | Remove, and note the reason category if known |
The common thread is that the person in the data made a choice after you collected it. A dataset that keeps the old version indefinitely overrides that choice, which is the core of the ethical question, and often of the contractual one too.
Where the obligations come from
The rules for a given dataset usually come from several places at once, and the strictest one applies in practice:
- Developer or API terms. If the data came through an official API, its terms commonly require removing content the author has deleted or made private, sometimes within a stated period. This is often the most specific rule a project faces.
- The platform's general terms, which can limit how public content may be stored and redistributed even when it was collected from public pages.
- Privacy law. In Europe, the GDPR gives people a right to have their personal data erased in certain circumstances, with exceptions that can include some research and archiving uses. Other jurisdictions have their own rules.
- Research ethics approval or the data management plan, which may already state how deletions will be handled.
- Publication and data sharing rules set by journals, funders or clients.
Read the applicable terms for each platform separately, because they differ and change. When a project depends on an exception, such as a research exemption under privacy law, the question of whether it applies is one for a lawyer or the institution's data protection officer, not something to settle by assumption.
Three ways to detect deletions and edits
Detecting change is the hard part, because a deleted post sends no signal on its own. There are three approaches, compared here against the same criteria: coverage, cost, how quickly changes are noticed, and fit with the terms.
Re-checking through the official API
Many APIs let you look up posts by identifier in batches. Re-requesting the identifiers in your dataset on a schedule shows which posts are no longer available and returns the current text of the ones that remain. Coverage is good for anything collected through that API, the cost is a predictable number of requests counted against your quota, and the terms usually expect exactly this. The limit is the quota itself: a large dataset may need weeks to cycle through, so prioritize recent posts and the ones you plan to quote or publish.
Deletion notices from the platform
Some platforms provide a way to receive notices of deleted or changed content, or a compliance endpoint that lists affected identifiers. Where it exists, this is the most efficient route, since you hear about changes without re-checking everything. Coverage depends entirely on what the platform offers and to whom, so confirm in the documentation whether it is available for your access level.
Re-visiting public pages
For posts collected from public pages, the only option may be to request the page again and see whether it still exists. This is the heaviest route for both sides, so it should run at a slow pace, follow the platform's robots.txt and terms, and cover only the posts you actually need to verify. A post that now requires a login to view has stopped being public, and the right reading of that is the same as a deletion: the content leaves the dataset. Trying to reach it some other way would defeat the purpose of the check.
For most projects, the order of preference is notices first, API re-checks second and page visits only where neither exists. The comparison of official APIs and scraping public pages for social media data discusses the wider choice of access route.
Setting the re-check schedule
The schedule should start from the strictest deadline in your terms and work backwards. If a platform's terms ask for removal within a set number of days, the full cycle of re-checks has to finish well inside that window, with room for failed runs. If no deadline is stated, a cycle that matches how the data is used is a reasonable default: before each analysis round, and always before publication.
Quotas decide what is feasible. Suppose, hypothetically, an API allows looking up 100 posts per request and a project may spend 5,000 requests a day on re-checks. That covers 500,000 posts a day, so a dataset of a few hundred thousand posts can be cycled daily. A dataset of 20 million posts would take more than a month per cycle, which may be too slow for a short removal deadline. The usual answer is tiering: re-check recent posts often, since they are the ones most likely to still change, re-check older posts less often, and always re-check the specific posts that will be quoted, shown or shared.
Whatever the schedule, track two numbers per cycle: how many posts were checked, and how many were found removed or changed. A cycle that checks far fewer posts than planned, or suddenly finds no removals at all, usually means the re-check failed rather than that nothing changed.
What to do with each change
Once a change is detected, the handling should be mechanical, written down in advance and applied the same way every time.
Deleted posts and private accounts: remove the text, media references, and any personal fields such as names, handles and profile descriptions. If your terms allow it, keep a minimal tombstone record, holding the identifier, a status and the date the change was detected, so that later runs do not collect the post again and so that counts can be explained. Some terms do not allow keeping even identifiers; check before designing around tombstones.
Edited posts: replace the stored text with the current version and update a last-checked date. Keeping a history of versions is common practice for product data, but for content written by people it needs a specific justification, such as a study of how misinformation is corrected, and an explicit basis in the terms and the data plan.
Platform removals: handle them like deletions. If the reason category is known and matters to the analysis, such as a study of moderation, record the category without keeping the content.
Suppose, hypothetically, a team holds 120,000 public posts collected over two months, and its first full re-check finds that a few thousand are no longer available and a smaller number have been edited. The deletion step removes the content of the unavailable posts and leaves tombstones, the edit step updates the changed ones, and the team records both numbers in its log. If the share of unavailable posts in one topic is much higher than in others, that is itself a finding to note, because the remaining sample may no longer represent that topic in the same way.
Keep a removal log separate from the content: the date of each cycle, the counts by kind of change, and the rule applied. It holds no posts and no names, so it can be kept for the life of the project, and it answers the questions a reviewer, an editor or a platform is most likely to ask: whether removals were handled, how quickly, and how they affected the sample.
Derived data, backups and published work
Removal from the main table is the easy part. Copies of the content tend to spread:
- Analysis files and extracts on people's machines, which should be regenerated from the main table rather than kept.
- Annotation and coding files, where text is often copied next to the labels. Store the identifier and the label, and join to the text when needed.
- Aggregates such as counts per day or per topic. These normally contain no content and can usually stay, though it is worth stating that choice in the data plan.
- Backups, which cannot always be edited. Set a retention period after which old backups expire, so deleted content does not survive indefinitely.
- Quotations in reports and papers. Before publishing, check that each quoted post is still available, and consider paraphrasing so that individuals cannot be found by searching the exact words.
Shared datasets raise the same question in a different form. Sharing post identifiers instead of full content lets each recipient retrieve only what is still available, which is why some platforms have required that approach.
How the choice differs by reader
Academic researchers should write the deletion and edit rules into the data management plan before collection, run API re-checks on a schedule, and re-check every quoted post before publication. If the study depends on content that authors later remove, discuss it with the ethics board early, not at publication time.
Brand and market analysts usually work with aggregates and short time windows. A monthly re-check and removal cycle, plus keeping reports at the level of counts and themes rather than individual posts, covers most needs with little effort.
Product teams that show social content in an application, such as a feed of public reviews or mentions, need the fastest cycle, since displaying a post its author deleted is visible to everyone. Platform notices, where offered, are the right tool here, with frequent re-checks as a fallback.
Teams that outsource collection should put the deletion schedule into the agreement. Whether the provider is ScrapingIsNotACrime or another service, the contract should say how removed content is detected, how quickly deliveries reflect it, and how the client is told which records to remove from copies already delivered. The same expectations appear in ethical and effective social media scraping best practices.
Keeping social media datasets in sync with deletions and edits
The practical routine fits on one page. Before collecting, find the deletion rules in each platform's terms and write your own handling for the five kinds of change. During the project, detect changes through notices or API re-checks on a fixed schedule, remove content mechanically, keep tombstones only where allowed, and log the numbers. Keep content out of derived files, let backups expire, and re-check anything you quote before it is published. A dataset maintained this way respects the choices of the people in it, and it is also easier to defend when someone asks how it was handled.