Search the Community
Showing results for tags 'web scraping'.
Hi my name is John, and i'm working in a project where i need to get data from a table inside a website., can someone help me?
Hi i want to know how to use http get command in MEP for active session of already logged in website. I have logged in to a website and have a result page there,can i use http get command to copy result to variable and then parse data
so i've had this mxpro software since about this past March 2014... i believe that it's the best thing since sliced bread and i've created some really powerful routines using it and have only recently found that this forum existed after searching for some assistance for some scenarios which i've 'hit a wall' with. i believe that i've exhausted the provided programming parameters of mxpro for a few scenarios which i've encountered. at this point, i've been learning a bit of java, c, and vb out of the need to expand the capabilities that i thought i might be able to utilize using the 'external script' function. i've searched this forum and others' for a solution but have not found anything which satisfies the requirements of what i envision. having said that, please forgive me if i may have overlooked a similar thread which has already covered this topic. I'm looking to automate the entry of contents into a web form coded in html/java... fields and button(s). i have seen routines in java and vba which can manipulate the data as i envision it. thus, i thought that i could implement those scripts into my routine to manipulate the form as well....however, it seems that there are specific parameters which the external script command can accommodate. for the record, i have successfully created a mxpro routine which does exactly what i want; however, since there are a few mouse movements involved, the routine will not execute reliably if the window is not sized properly, or if there are different text sizes or zoom properties. and tabbing thru fields DOES work, however, the webpage reacts differently if there are values entered into some fields and thus the amount of tabs can differ based upon what fields have values and those that do not. so the Tab order can change. what i need to do is 3 things(in Internet Explorer): 1. copy the source code of a webpage(which i have already successfully done using right click and moving down the right click menu to 'view source' - there are frames in the page so ctrl>U doesn't work); additionally, i would like this to occur without opening a separate view source window; 2. be able to enter data directly into a form field based upon its field ID/name/value(in other words, not using Tab to move to the field to manipulate its contents); 3. be able to click on a 'submit' button using its name/ID/value(not using the mouse). -the above, all without using the keyboard or mouse. ultimately, i need the routine to be scalable... meaning whatever the user preferences are(i.e. Internet Explorer zoom setting, window size, text size, etc etc) the routine needs to still complete as programmed. i figure that using other programming languages which are more suitable to manipulating web pages and its contents would only expand the ability that mxpro routines can accomplish. i have downloaded, viewed, and ran the external scripts which the gentleman from Insight wrote years ago which all work fine... however, creating my own scripts yielded ONLY the simple "Hello World" output to console from VB's 'MsgBox("Hello World!")' command to print to the console... console.log("Hello World") doesn't output, nor does System.out.println("Hello World"); anything that i have tried does not log to the console or function as it would in an API. i have changed the output type or language type to match the entered type within the 'external script' command. i tried these simple examples to test the external script function... let alone more complex routines to complete the 3 scenarios i outlined above. any suggestions are appreciated and sorry for the long post! just wanted to be thorough. i'm sure more info may be necessary which i can provide.