python - Fetching Image from URL using BeautifulSoup -



python - Fetching Image from URL using BeautifulSoup -

i trying fetch of import images , not thumbnail or other gifs wikipedia page , using next code. "img" coming length of "0". suggestion on how rectify it.

code :

import urllib import urllib2 bs4 import beautifulsoup import os html = urllib2.urlopen("http://en.wikipedia.org/wiki/main_page") soup = beautifulsoup(html) imgs = soup.findall("div",{"class":"image"})

also if can explain in detail how utilize findall looking @ "source element" in webpage. awesome.

the a tags on page have image class, not div:

>>> img_links = soup.findall("a", {"class":"image"}) >>> img_link in img_links: ... print img_link.img['src'] ... //upload.wikimedia.org/wikipedia/commons/thumb/1/1f/stora_kronan.jpeg/100px-stora_kronan.jpeg //upload.wikimedia.org/wikipedia/commons/thumb/4/4b/christuss%c3%a4ule_8.jpg/77px-christuss%c3%a4ule_8.jpg ...

or, better, utilize a.image > img css selector:

>>> img in soup.select('a.image > img'): ... print img['src'] //upload.wikimedia.org/wikipedia/commons/thumb/1/1f/stora_kronan.jpeg/100px-stora_kronan.jpeg //upload.wikimedia.org/wikipedia/commons/thumb/4/4b/christuss%c3%a4ule_8.jpg/77px-christuss%c3%a4ule_8.jpg ...

upd (downloading images using urllib.urlretrieve):

from urllib import urlretrieve import urlparse bs4 import beautifulsoup import urllib2 url = "http://en.wikipedia.org/wiki/main_page" soup = beautifulsoup(urllib2.urlopen(url)) img in soup.select('a.image > img'): img_url = urlparse.urljoin(url, img['src']) file_name = img['src'].split('/')[-1] urlretrieve(img_url, file_name)

python url web-scraping beautifulsoup urllib

Comments

Post a Comment

Popular posts from this blog

php - Android app custom user registration and login with cookie using facebook sdk -

php - Android app custom user registration and login with cookie using facebook sdk - i developing android application in users need register , log in using facebook. approximately, flow. open app first time, authenticate facebook, data, send info server create them business relationship in application, start session new business relationship in server, cookie perform later interactions. so far good. what need know wich info have send server, can able identify users when log in seccond time. because need start session in server , send cookie android app farther interactions. i'm using php server side language. i thinking getting getaccesstoken() in android facebook sdk, pass server on business relationship creation , storing in android app. but can't manage how login users on sec time utilize app. any suggestions ? in advance ! to uniquely identify user within app, should create request /me , user's id. guaranteed unique , remain con...
Read more

c# - Create a Notification Object (Email or Page) At Run Time -- Dependency Injection or Factory -

c# - Create a Notification Object (Email or Page) At Run Time -- Dependency Injection or Factory - i have code emails me notification when happens. trying add together page , also remove dependency on email. for reference; have name of notification want send stored string in database. @ runtime have list of people , string representation of want receive. i thought dependency injection ninject, kernel bindings got big , seemed hacking together. then thought simple factory, activator.createinstance(email/page) inotifcation, don't know how handle differing constructor parameters (unless utilize common, complex property, employee, has info need). so far have this: public interface inotification { void sendmessage(); } public class email: inotification { private readonly string _toemailaddress; private string body { { homecoming "email"; } } private static string smtphost { ...
Read more

Set Up Of Common Name Of SSL Certificate To Protect Plesk Panel -

Set Up Of Common Name Of SSL Certificate To Protect Plesk Panel - a pci compliance scanner balking self signed ssl certificate protecting secure access plesk panel contains name mismatch between location of plesk panel , name on certificate, namely self-signed cert's name "parallels" , domain reach plesk 'ip address:8443'. so figured go ahead , free ssl certificate seek fiddle error. when generated certificate used server domain name site name when generated certificate. if visit 'domain name:8443' fine, no ssl warning. if visit 'ip address:8443' (which believe scanner does) certificate name mismatch error, digicert's ssl checker says certificate name should ip address. can generate certificate mutual name ip address? tempted should pci scanner accepts, right mutual name use? run issue before? you can seek subject alternative name extension ip address. also, can maintain "parallels" mutual name, , add together ...
Read more