Skip to content

fix(linkedin): handle None job_level when LinkedIn omits "Seniority level"#354

Open
Recursing wants to merge 1 commit into
speedyapply:mainfrom
Recursing:fix/linkedin-job-level-none
Open

fix(linkedin): handle None job_level when LinkedIn omits "Seniority level"#354
Recursing wants to merge 1 commit into
speedyapply:mainfrom
Recursing:fix/linkedin-job-level-none

Conversation

@Recursing

Copy link
Copy Markdown

Bug

linkedin/__init__.py:240 raises AttributeError: 'NoneType' object has no attribute 'lower' when parse_job_level (util.py:42) returns None — which it correctly does (per its -> str | None signature) when the LinkedIn detail page lacks a "Seniority level" criterion. Common on non-US/UK postings (NL, DK, SE).

dict.get(k, default) only uses default when the key is missing, not when the value is None. So .get("job_level", "") returns None, and None.lower() raises.

The error aborts the entire scrape_jobs call, losing all results — including from other sites (Indeed, etc.) in the same invocation.

Reproduction

from bs4 import BeautifulSoup
from jobspy.linkedin.util import parse_job_level

# Detail page with Employment type + Industries but no "Seniority level"
soup = BeautifulSoup(
    '<h3 class="description__job-criteria-subheader">Employment type</h3>',
    "html.parser",
)
assert parse_job_level(soup) is None
{"job_level": parse_job_level(soup)}.get("job_level", "").lower()
# AttributeError: 'NoneType' object has no attribute 'lower'

Impact

In a daily LinkedIn pipeline with linkedin_fetch_description=True, ~15–30% of queries abort this way. From one run:

WARNING jobspy: query 'head of innovation'/'Rotterdam'  failed: 'NoneType' object has no attribute 'lower'
WARNING jobspy: query 'circular economy'/'Netherlands'  failed: 'NoneType' object has no attribute 'lower'
WARNING jobspy: query 'head of innovation'/'Copenhagen' failed: 'NoneType' object has no attribute 'lower'

Fix

(job_details.get("job_level") or "").lower() — handles both missing key and None value, preserves the existing , "" intent. Doesn't touch parse_job_level's honest str | None signature; the call site is the bug.

Confirmed on python-jobspy==1.1.82 (latest on PyPI) and main at fda080a.


🤖 Written by Claude Code.

…evel"

parse_job_level returns None (per its `-> str | None` signature) when the
LinkedIn detail page lacks a "Seniority level" criterion. dict.get(k, "")
only substitutes the default for missing keys, not None values, so
.get("job_level", "").lower() raises AttributeError, aborting the entire
scrape_jobs call (including results from other sites in the same call).

Use `(... or "")` to handle both missing key and None value while
preserving the existing default-to-empty intent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant