Search on blog:

Python: How to use Selenium with local HTML in string.

To run Selenium on local HTML which you have in string you can use

driver.get("data:text/html;charset=utf-8," + html)

Full example

html = '''
<ul>
  <li>Contains Enzymatically Active B-Vitamins
  </li>
  <li>Dietary Supplement
  </li>
  <li>Non-GMO LE Certified
  </li>
</ul>'''

import selenium.webdriver

driver = selenium.webdriver.Firefox()

driver.get("data:text/html;charset=utf-8," + html)

elements = driver.find_elements_by_tag_name('li')

elements = [i.text for i in elements]

print( ", ".join(elements)) 

Selenium: Jak zamknąć alert stworzony przez JavaScript

JavaScript może tworzyć trzy standarowe wyskakujące alerty: alert(), confirm() lub prompt().

  • wszystkie z nich mają przycisk OK
  • confirm() i prompt() mają przycisk CANCEL
  • prompt() ma pole tekstowe

Aby wcisnąć OK

driver.switch_to.alert.accept()   # press 'OK'

Aby wcisnąć CANCEL (tylko w confirm() i prompt())

driver.switch_to.alert.dismiss()   # press 'Cancel …

Selenium: Wysłanie zawartości schowka do pola w przeglądarce

Gdy znajdziesz pole tekstowe na stronie wtedy możesz wysłać Ctrl+V aby wstawić tekst ze schowka do tego pola.

import selenium.webdriver
from selenium.webdriver.common.keys import Keys 

driver = selenium.webdriver.Firefox()
driver.get('https://google.com')

item = driver.find_element_by_name('q')
item.send_keys(Keys.CONTROL + "v")
#item.send_keys(Keys …

Selenium: How to login using css_selector

It uses css selector to find element.

import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get('http://quotes.toscrape.com/login')

item = driver.find_element_by_css_selector('input[id="username"]')
item.send_keys('Hello World!')

item = driver.find_element_by_css_selector('input[id="password"]')
item.send_keys('Hello World!')

item = driver.find_element_by_css_selector('input[value="Login"]')
item.click()

I …

Selenium: Jak wykonać logowanie z uzyciem css_selector

Przykład używa css selector do znalezienia elementu.

import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get('http://quotes.toscrape.com/login')

item = driver.find_element_by_css_selector('input[id="username"]')
item.send_keys('Hello World!')

item = driver.find_element_by_css_selector('input[id="password"]')
item.send_keys('Hello World!')

item = driver.find_element_by_css_selector('input[value="Login"]')
item.click()

Używam …

Scraping incomplete data with Selenium

Sometimes people scrape separatelly different values from page

all_names  = driver.find_elements_by_xpath('.//h3/a')
all_prices = driver.find_elements_by_class_name('price_color')
all_others = driver.find_elements_by_class_name("other")

and later group them using zip()

for row in zip(all_names, all_prices, all_others):
    print(row)

but it makes problem if for some items data are incomplete - like other in …

Scraping niekompletnych danych z Selenium

Czasami ludzie zbierają oddzielnie różne wartości ze strony

all_names  = driver.find_elements_by_xpath('.//h3/a')
all_prices = driver.find_elements_by_class_name('price_color')
all_others = driver.find_elements_by_class_name("other")

a potem grupują ją używając zip()

for row in zip(all_names, all_prices, all_others):
    print(row)

ale to może powodować problem jeśli niektóre elementy mają niekompletne dane - jak other w …

Python: How to scrape aastocks.com with requests

It is example code to scrape it:

# date: 2019.09.16
# https://stackoverflow.com/questions/57861715/scrapy-infinite-scrolling-no-pagination-indication
# http://www.aastocks.com
import requests

newstime = '934735827'
newsid = 'HKEX-EPS-20190815-003587368'

url = 'http://www.aastocks.com/tc/resources/datafeed/getmorenews.ashx?cat=all&newstime={}&newsid={}&period=0&key=&symbol=00001'
url_artickle = "http://www.aastocks …

Python: How to scrape allegro.pl with scrapy

It is example code to scrape it:

# date: 2017.12.10
# https://stackoverflow.com/a/47744135/1832058

import scrapy

#from allegro.items import AllegroItem

#class AllegroItem(scrapy.Item):
#    product_name = scrapy.Field()
#    product_sale_price = scrapy.Field()
#    product_seller = scrapy.Field()

class AllegroPrices(scrapy.Spider):

    name = "AllegroPrices"
    allowed_domains = ["allegro.pl"]

    start_urls = [
        "http://allegro.pl …

Python: How to scrape alloschool.com with scrapy

It is example code to scrape it:

#!/usr/bin/env python3

# date: 2019.07.29
# https://stackoverflow.com/questions/57245315/using-scrapy-how-to-download-pdf-files-from-some-extracted-links

import scrapy

class MySpider(scrapy.Spider):

    name = 'myspider'

    start_urls = [
          'https://www.alloschool.com/course/alriadhiat-alaol-ibtdaii',
    ]

    def parse(self, response):

        for link in response.css('.default .er').xpath('@href').extract …

Python: How to scrape amazon.com (2) with selenium

It is example code to scrape it:

#!/usr/bin/env python3

# date: 2020.03.30

import selenium.webdriver

url = 'https://www.amazon.com/international-sales-offers/b/?ie=UTF8&node=15529609011&ref_=nav_cs_gb_intl'

driver = selenium.webdriver.Firefox()
driver.get(url)

for x in range(10):
    deal = driver.find_element_by_id('100_dealView_' + str(x))

    image …

Python: How to scrape aopa.org with selenium

It is example code to scrape it:

# https://stackoverflow.com/questions/60601053/python-selenium-for-loop-iterates-through-entire-website/60601428

from selenium import webdriver
import time

driver = webdriver.Chrome()

#wait = WebDriverWait(driver, 10)

driver.get("https://www.aopa.org/destinations/airports/state/AL")
time.sleep(3)

airport_list = []
paved_runway = []

airport_row = driver.find_elements_by_xpath('//div[@class = "state-airports__airport"]')
print(len …

Python: How to scrape api.weatherflow.com with requests

It is example code to scrape it:

#!/usr/bin/env python3

# date: 2020.02.10
# 

import requests

url = 'https://api.weatherflow.com/wxengine/rest/model/getModelDataBySpot?model_id=-1&spot_id=110&units_wind=mph&units_temp=F&format=json&wf_apikey=84e778ae-fe8e-4b8f-8d33-6bc88967a2b1&wf_token=f147702351af100d7c220b633d085318&v=1.1'
r = requests.get(url)
data = r.json …

Python: How to scrape apps.upenn.edu with selenium

It is example code to scrape it:

#!/usr/bin/env python3

# date: 2020.02.26
# 

import selenium.webdriver

def scrape(last_name, first_name):        
    url = 'https://directory.apps.upenn.edu/directory/jsp/fast.do'

    driver = selenium.webdriver.Firefox()
    driver.get(url)

    inputs = driver.find_elements_by_tag_name('input')

    #for item in inputs:
    #    print(item.get_attribute …

Python: How to scrape associatedrealtorsaruba.com with requests

It is example code to scrape it:

#!/usr/bin/env python3

# date: 2020.01.07
# https://stackoverflow.com/questions/59632031/how-to-extract-href-when-href-element-is-a-hyperlink?noredirect=1#comment105434826_59632031

import requests
from bs4 import BeautifulSoup as BS

url = 'https://associatedrealtorsaruba.com/index.php?option=com_ezrealty&Itemid=11&task=results&cnid=0&custom7=&custom8=&parking=&type …

« Page: 1 / 12 »