selenium-sentinel

Replace time.sleep() with WebDriverWait and expected_conditions such as element_to_be_clickable or visibility_of_element_located, waiting for the state the next action needs. Keep implicit waits at zero and never mix them with explicit waits, since combining them produces unpredictable timeouts.

Define a locator contract and column contract, then paginate by URL or Next-button clicks while waiting for staleness of the old rows. Persist records page by page to JSONL with a checkpoint so a crash costs one page, and verify the record count against the site's announced total.

Yes, use webdriver.Remote with W3C Options against a trusted Grid endpoint over HTTPS. Each worker needs a unique profile, download directory, and output sink, and uploads from the client require setting driver.file_detector to LocalFileDetector.

It occurs when the DOM is rebuilt after navigation, refresh, or AJAX replacement, invalidating previously located elements. Re-locate elements after any DOM-changing action, and after a click that triggers a re-render, wait for staleness_of the old element before finding its replacement.

Poll the isolated download directory on the filesystem: confirm partial files like .crdownload or .part are gone, the size is stable, and the name, extension, and content signature match expectations. A click returning without an exception proves nothing about the file.

Avoid Selenium when an official API, bulk export, static HTTP response, or the page's own JSON endpoint can provide the data within your authorization. Selenium is justified only when rendering, authenticated UI state, or real interaction is required, and never for bypassing CAPTCHAs, rate limits, or access controls.