Selenium can't seem to find clickable button - Python

Question

I'm trying to download data as csv from this website and it seems easy enough but selenium is unable to locate the button. I have downloaded page source and verified that it is there, so i am unsure what the issue is.

This is the element -

<button class="btn exportBTN btn-default" id="_eucleflegislationlist_WAR_euclefportlet_exportButtonCSV" name="_eucleflegislationlist_WAR_euclefportlet_exportButtonCSV" onclick="_eucleflegislationlist_WAR_euclefportlet_exportData('csv');" type="button">
        <span class="lfr-btn-label">CSV</span>
        </button>

And this is my code:

# Set up the Chrome WebDriver
driver = webdriver.Chrome(options=chrome_options)

# Open the webpage
driver.get('https://www.echa.europa.eu/web/guest/cosmetics-prohibited-substances?p_p_id=eucleflegislationlist_WAR_euclefportlet&p_p_lifecycle=0&p_p_state=normal&p_p_mode=view&_eucleflegislationlist_WAR_euclefportlet_cur=1&_eucleflegislationlist_WAR_euclefportlet_substance_identifier_field_key=&_eucleflegislationlist_WAR_euclefportlet_delta=50&_eucleflegislationlist_WAR_euclefportlet_doSearch=&_eucleflegislationlist_WAR_euclefportlet_deltaParamValue=50&_eucleflegislationlist_WAR_euclefportlet_orderByCol=fld_erc2_maxthres&_eucleflegislationlist_WAR_euclefportlet_orderByType=desc')

# Optional: Wait for the page to load
time.sleep(5)

# Step 1: Click the "Accept Disclaimer" button
accept_button = driver.find_element(By.ID, "_viewsubstances_WAR_echarevsubstanceportlet_acceptDisclaimerButton")
accept_button.click()

# Optional: Wait for any page transition after accepting terms
time.sleep(3)

# Step 2: Find the CSV export button and click it to download
csv_button = driver.find_element(By.ID, "_euclegislationlist_WAR_euclefportlet_exportButtonCSV")
csv_button.click()

# Optional: Wait for the download to complete
time.sleep(10)

# Close the browser
driver.quit()

Error message:

raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="_euclegislationlist_WAR_euclefportlet_exportButtonCSV"]"} (Session info: chrome=129.0.6668.90); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception

Instead of saying vague things like "selenium is unable to locate the button", please show us the actual error message. — John Gordon
– John Gordon, Commented Oct 2, 2024 at 3:31
I think you misspelled the item id. Your code is looking for _euclegislationlist..., but the actual html id is _eucleflegislationlist.... — John Gordon
– John Gordon, Commented Oct 2, 2024 at 4:32

chitown88 · Accepted Answer · 2024-10-02 16:16:48Z

Do you need to use Selenium? Bit of an over kill since the table is in the html.

import requests
import pandas as pd


page=0
dfs=[]
while True:
    page+=1
    url = 'https://www.echa.europa.eu/web/guest/cosmetics-prohibited-substances'
    payload = {
        'p_p_id': 'eucleflegislationlist_WAR_euclefportlet',
    'p_p_lifecycle': '0',
    'p_p_state': 'normal',
    'p_p_mode': 'view',
    '_eucleflegislationlist_WAR_euclefportlet_cur': '1',
    '_eucleflegislationlist_WAR_euclefportlet_orderByCol': 'fld_erc2_maxthres',
    '_eucleflegislationlist_WAR_euclefportlet_substance_identifier_field_key': '',
    '_eucleflegislationlist_WAR_euclefportlet_orderByType': 'desc',
    '_eucleflegislationlist_WAR_euclefportlet_doSearch': '',
    '_eucleflegislationlist_WAR_euclefportlet_deltaParamValue': '50',
    '_eucleflegislationlist_WAR_euclefportlet_resetCur': 'false',
    '_eucleflegislationlist_WAR_euclefportlet_delta': '200',
    '_eucleflegislationlist_WAR_euclefportlet_cur': f'{page}'}



    response = requests.get(url, params=payload)
    df = pd.read_html(response.text)[0]

    if len(df) == 1:
        break
    dfs.append(df)
    print(f'Page: {page}')
    


df = pd.concat(dfs)
df = df.dropna()
df.to_csv('output.csv', index=False)

Output:

Substance Name     EC No.    CAS No.  Ref No.                                           Product type, body parts Maximum Threshold                                                                                                                                                                                                            Restriction(s) Unnamed: 7
0            Safrole  202-345-4    94-59-7    360.0  All cosmetic products except products for dental and oral hygiene           100 ppm                                                                       Not permitted except for normal content in the natural essences used and provided the concentration does not exceed 100 ppm in the finished product    Details
1            Safrole  202-345-4    94-59-7    360.0                               Products for dental and oral hygiene            50 ppm  Not permitted except for normal content in the natural essences used and provided the concentration does not exceed 50 ppm in the finished product and provided that is not present in toothpastes intended for children    Details
2        Trioxysalen  223-459-0  3902-71-4    358.0                      Sun protection products and bronzing products           1 mg/kg                                                                                                                                                    Not permitted for all products except if occurring in natural essences    Details
3     Furocoumarines          -          -    358.0                      Sun protection products and bronzing products           1 mg/kg                                                                                                                                                    Not permitted for all products except if occurring in natural essences    Details
4  8-Methoxypsoralen  206-066-9   298-81-7    358.0                      Sun protection products and bronzing products           1 mg/kg                                                                                                                                                    Not permitted for all products except if occurring in natural essences    Details
....

[666 rows x 8 columns]

Collectives™ on Stack Overflow

Selenium can't seem to find clickable button - Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related