# Step 1: Gather Data

Gathering data is the most important step in solving any supervised machine
learning problem. Your text classifier can only be as good as the dataset it is
built from.

If you don't have a specific problem you want to solve and are just interested
in exploring text classification in general, there are plenty of open source
datasets available. You can find links to some of them in our [GitHub
repo](https://github.com/google/eng-edu/blob/master/ml/guides/text_classification/load_data.py).
On the other hand, if you are tackling a specific problem,
you will need to collect the necessary data. Many organizations provide public
APIs for accessing their data---for example, the
[X API](https://developer.x.com/docs) or the
[NY Times API](http://developer.nytimes.com/). You may be able to leverage
these APIs for the problem you are trying to solve.

Here are some important things to remember when collecting data:

- If you are using a public API, understand the *limitations* of the API before using them. For example, some APIs set a limit on the rate at which you can make queries.
- The more training examples (referred to as *samples* in the rest of this guide) you have, the better. This will help your model [generalize](https://developers.google.com/machine-learning/glossary#generalization) better.
- Make sure the number of samples for every *class* or topic is not overly [imbalanced](https://developers.google.com/machine-learning/glossary#class_imbalanced_data_set). That is, you should have comparable number of samples in each class.
- Make sure that your samples adequately cover the *space of possible inputs*, not only the common cases.

Throughout this guide, we will use the [Internet Movie Database (IMDb) movie
reviews dataset](http://ai.stanford.edu/%7Eamaas/data/sentiment/) to illustrate
the workflow. This dataset contains movie reviews posted by people on the IMDb
website, as well as the corresponding labels ("positive" or "negative")
indicating whether the reviewer liked the movie or not. This is a classic
example of a sentiment analysis problem.