hawk1ns Posted July 13, 2003 Posted July 13, 2003 Hello , a short description of what i want to do : a friend has asked me to try this for him.. i have this working on a manual basis , i.e using buttons and manually inputting product number so i am nearly there i input a number in a textbox ( item number ) then click browse button on application and the website appears in a axwebbrowser1 window i then click on a view source btn and can view the html in a textbox i can the sort and parce through the html to grab product info and image url etc i will store this in a database when i am finished :) the above works fine ... but the idea is to set a loop from 1 to say 100000 and have the pages entered automaticaly.... heres the problem tho: the loop goes way to fast and does not allow the web page to load before continuing to next step.. how can i enter a check so that everything stops untill the web page has fully loaded ? Kind regards Carl Quote
Leaders dynamic_sysop Posted July 13, 2003 Leaders Posted July 13, 2003 if you only want to be able to navigate to the sites and retrieve the source of them sites / also be able to download images etc... why not use System.Net.WebClient ? here's an example , just downloading 3 sites , with a 100 millisecond pause : Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click Dim strUrls() As String = "http://google.com [url]http://yahoo.com[/url] [url]http://msn.com[/url]".Split(" ") Dim x(3) As System.Net.WebClient Dim i As Integer For i = 0 To 2 x(i) = New System.Net.WebClient() Dim st As New IO.StreamReader(x(i).OpenRead(strUrls(i))) MsgBox(st.ReadToEnd) Threading.Thread.Sleep(100) st.Close() x(i).Dispose() Next End Sub Quote
hawk1ns Posted July 13, 2003 Author Posted July 13, 2003 thanks for the suggestions :) I seem to have a problem with this line of the code : Dim strUrls() As String = "http://google.com <a href="http://yahoo.com" target="_blank">http://yahoo.com</a> <a href="http://msn.com" target="_blank">http://msn.com</a>".Split(" ") blue error lines on a few things can you suggest why ? Kind Regards Carl Quote
Leaders dynamic_sysop Posted July 13, 2003 Leaders Posted July 13, 2003 yes it's supposed to be spaces not the < href stuff , thats how it converted on the thread :-\ http://google.com http://yahoo.com http://msn.com with spaces. Quote
hawk1ns Posted July 13, 2003 Author Posted July 13, 2003 well i think i need to keep going the way i was , cos when i get the html by the way above i get a different page , it also says : turn your browser scripting on. :) cheers Quote
hawk1ns Posted January 27, 2004 Author Posted January 27, 2004 grabbing the url hello , i have used the script above to display the html text in a textbox , i would now like to also put the actual url into this box. basically i have some links for a php script on my website that used numbers in its directory , to try and catalogue i want to know where each link goes .. for example http://somesite.com/phpscript?id=number actually goes to a website so the html i get from the code above displays the html code for the page but i also want to grab the url web address? Kind Regards Carl Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.