Cody Arlie ReedResearch + data
WorkHow well do job descriptions predict occupation?

Natural language processing · machine learning

How well do job descriptions predict occupation?

ProblemSurveys must turn brief, uneven descriptions of people's work into standard occupation codes, but the text may not distinguish occupations reliably.

What I builtI represented short job descriptions with Word2Vec, trained five feed-forward neural networks in TensorFlow to rank 120 broad occupation groups, and audited their errors and predicted probabilities.

OwnershipSole researcher and developer: study design, TensorFlow modeling, evaluation, visualization, and writing

Result

The model ranked the human-assigned code first 59.7% of the time and among its first two choices 74.7% of the time, with large differences across occupations.

Classification accuracy showing a 5.5 percent baseline, 59.7 percent first-choice accuracy, 74.7 percent first-or-second-choice accuracy, and large differences across occupations
Accuracy ranged from 96.2% for teachers to 40.0% for top executives

Result

Predictability varied sharply by occupation.

On 1,032 job descriptions excluded from model training, the assigned three-digit Standard Occupational Classification (SOC) code was the model’s first choice 59.7% of the time. It appeared among the first two choices 74.7% of the time. A classifier that always chose the largest category would score 5.5%. Agreement means that the model’s ranked code matched the human-assigned code.

Inputs

Survey answers and official occupation definitions serve different purposes.

The Education Longitudinal Study (ELS) records a short job title and the main duties. The Standard Occupational Classification (SOC) defines the target category. The model predicts the human-assigned SOC code from the survey text.

ELS survey response
Prompt 1
What is the job called?
Prompt 2
What are the main duties?

Brief everyday language, sometimes supplied by another household member.

SOC reference text
Input 1
Occupational title
Input 2
Formal definition

Standardized language written to distinguish occupations.

Why the survey descriptions vary

The respondent source changes across survey waves and missing-data conditions. A description may come from a parent, a young person describing a parent, or an imputed (filled-in) fallback. Standardized SOC title-and-definition text helped fit the Word2Vec vocabulary and also entered the training data as labeled examples for the corresponding SOC codes.

Model

A compact deep-learning classifier ranked 120 occupation groups.

Each description became a 256-value vector by averaging its Word2Vec word embeddings. Each TensorFlow neural network used two dense hidden layers—512 and 256 units—with dropout and a 120-category output. I trained five networks and averaged their predicted probabilities to produce the final rankings.

Occupation patterns

Distinctive occupations were easier to classify.

First-choice agreement reached 96.2% for teachers and 84.2% for healthcare diagnosing or treating practitioners. Construction trades also reached 77.1%. At the other end, information and record clerks reached 43.3% and top executives 40.0%.

Broad professional groups averaged 63.0%, compared with 57.7% for other occupations. Performance did not rise steadily across the five-level O*NET Job Zone scale, which summarizes the education, experience, and training an occupation typically requires. Occupations with standardized titles and distinctive task language were easier to classify than broad roles such as executives and clerks.

Occupation coding

Occupation categories contain varied work.

ELS questionnaires collect short job titles and descriptions. O*NET combines surveys of people working in each occupation with occupational experts, analysts, job postings, and other sources to build standardized occupational information.

Martin-Caughey (2021) finds substantial variation within occupational categories. Occupations with cohesive names and clearer professional boundaries are easier to code; managers are especially difficult.