A common assumption in web data projects is that copyright only matters if you republish someone's article, and that collecting facts such as prices or opening hours raises no intellectual property questions at all. The first half is roughly right; the second breaks in places. Copyright generally protects original expression, not facts, so a price is usually free of copyright while a product description is usually not. But some jurisdictions, including the European Union and the United Kingdom, also protect databases themselves when someone invested substantially in building them, and that right can apply to collections of plain facts. This explainer defines the terms that come up most often, how they vary between jurisdictions and where they meet the other rules that govern scraping. It describes the general picture; whether a specific use is permitted is a question for a lawyer who knows the jurisdictions involved.
Copyright: protection for original expression
Copyright protects original works of authorship: text, photographs, illustrations, video, music and software, among others. On a typical website, that includes articles, product descriptions, reviews, images and the page's own design elements. Copyright arises automatically when a work is created; there is no need to register it or display a notice.
What copyright does not protect is just as important for data projects. Facts, ideas and data points are generally not protected as such. A product's price, a flight's departure time, a company's address and the temperature in a city are facts. The way someone wrote about those facts can be protected, but the facts themselves can usually be extracted and used.
A concrete contrast: collecting the price, size and availability of 500 products from a public catalog involves facts. Copying the 500 product descriptions and photographs into another website involves copyrighted works, and that is a very different situation, regardless of how the copying was done.
Originality and compilations
Originality is the threshold a work must meet to be protected. In the United States, the Supreme Court held in Feist Publications v. Rural Telephone Service in 1991 that a compilation of facts, in that case a telephone directory, is protected only to the extent its selection or arrangement is original, and that the effort spent gathering the facts does not by itself create copyright. Other countries set their own thresholds, and some have historically been more willing to protect effort.
A compilation, in this sense, is a collection of material selected or arranged in a way that may be original. A curated list of the best restaurants in a city, chosen and ordered according to someone's judgment, may have protectable selection. A complete list of every restaurant with a license, in alphabetical order, usually does not.
The database right: protection for investment
The sui generis database right exists in the European Union and, in a similar form, in the United Kingdom. It protects the maker of a database where there has been substantial investment in obtaining, verifying or presenting its contents, regardless of whether the database is original enough for copyright. It prevents the extraction or re-use of all or a substantial part of the database, judged by quantity or by importance, and repeated, systematic extraction of small parts can also be caught when it adds up to the same effect.
This right is the main reason the "facts are free" assumption breaks in Europe. A large, carefully maintained listing of public facts, such as a directory of businesses or a catalog of property listings, may be protected as a database even though each fact in it is not protected individually. The United States has no equivalent general database right, which is one of the clearest examples of how much the answer depends on jurisdiction.
A practical consequence: a project that collects a few fields from a European database for internal analysis is in a different position from one that copies most of the database to offer a competing service. The second is the situation the right was designed to address.
Exceptions: fair use, fair dealing and text and data mining
Copyright laws contain exceptions, and they vary widely. Fair use in the United States is a flexible defense assessed case by case against four factors: the purpose and character of the use, the nature of the work, the amount used and the effect on the market for the original. Transformative uses, such as analysis that does not substitute for the original, tend to weigh in favor of fair use, but no single factor decides the outcome.
Fair dealing, used in the United Kingdom and several other countries, is narrower and tied to specific purposes listed in the law, such as research, criticism or news reporting.
Text and data mining exceptions address the automated analysis of content specifically. The European Union has two: one for research organizations and cultural heritage institutions carrying out scientific research, and a broader one that applies to other users unless the rights holder has expressly reserved its rights; for content made publicly available online, that reservation is expected to be made in an appropriate way, such as by machine-readable means. The United Kingdom has an exception for text and data mining for non-commercial research. The details of who qualifies and how a reservation must be expressed are exactly the kind of point to confirm with a lawyer.
How these rights relate to terms of service and privacy law
Copyright and database rights are property rights in content. They sit alongside, and are separate from, two other sets of rules that often get mixed up with them.
- Terms of service are a contract question. A site's terms may restrict collection or reuse even of material that copyright does not protect, such as plain facts. Whether those terms bind a particular collector is its own question.
- Privacy law governs personal data, whatever its intellectual property status. A name and job title on a public profile may carry no copyright at all and still be fully subject to data protection rules.
- Licenses, such as Creative Commons licenses or open data licenses published by governments, grant permission in advance. Where a source offers its data under a license, the license terms, including attribution requirements, define what you may do.
Each of these has to be checked on its own. Passing one test says nothing about the others. The post on ethical data extraction and compliance looks at how the checks fit together in practice.
Common mistakes with copyright and scraped data
The first mistake is thinking that public means free to reuse. Content on a public page is visible to anyone, but copyright in it still belongs to its author or publisher. Visibility affects access, not ownership.
The second is thinking that collecting facts is always safe everywhere. In the European Union and the United Kingdom, extracting a substantial part of a protected database can be an issue even when every field is a fact. Projects that work with European sources should ask whether the source could be a protected database, and how much of it they take.
The third is focusing on collection and ignoring output. Many intellectual property questions turn on what happens after collection: storing full copies, republishing text or images, or selling the dataset. A project that stores only the facts it needs, discards copyrighted text it does not need and publishes aggregated results is in a much stronger position than one that keeps and redistributes everything.
The fourth is relying on a single label. Calling a project research, internal or non-commercial does not by itself bring it within an exception; each exception has conditions, and they differ by country. Whether a project is built in-house or delivered by a service such as ScrapingIsNotACrime, the decision about which content to keep and how to use it stays with the organization using the data, and that decision is the one to take to a lawyer when the sources are important or the use is commercial.
