Programmatic Web Search – Screen scraping – Web Page Input
08-Jan-1212 Leave a comment
To open a web page
Dim process3 As New Process
process3.StartInfo.FileName = Webpage
process3.Start()
To open a google search
use following plus search words separated by +
http://www.google.co.uk/search?q=
To read web page source – for screen scraping
Public Sub ReadWebPageSource()
Dim request As WebRequest = WebRequest.Create(Me.WebPage)
‘ If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials
‘request.Timeout = 3000
Try
‘ Get the response.
Dim response As HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)
‘ Display the status.
‘Console.WriteLine(response.StatusDescription)
‘ Get the stream containing content returned by the server.
Dim dataStream As Stream = response.GetResponseStream()
‘ Open the stream using a StreamReader for easy access.
Dim reader As New StreamReader(dataStream)
‘ Read the content.
Me.Source = reader.ReadToEnd()
‘ Display the content.
‘Console.WriteLine(responseFromServer)
‘ Cleanup the streams and the response.
reader.Close()
dataStream.Close()
response.Close()
Catch ex As Exception
Me.Source = “”
MsgBox(ex.Message)
End Try
End Sub
WebBrowser and Set Inner Text
tag id=”lst-ib” is Googles search text box
WebBrowser1.Navigate(“http://www.google.co.uk“)
WebBrowser1.Document.GetElementById(“lst-ib”).InnerText = InnerTextTextBox.Text
WebBrowser1.Document.GetElementById(“lst-ib”).Focus()