Hello v_pozidis,
There is an old post of mine that might help you a little to get started.
Of course, this post from 2015 is a little outdated and full of things I would not do like that today.
In general :
1- An HTTPGet request will give you the source code of the page : this is just text
2- Using specific HTML tag you will have to identify, you will look for the info you want embedded in the web page
3- Using Pos, Copy and so on, you can extract the data you need and save that data into variables
4- You can also use RegExp for finding your data, but it's a little tricky if you are not used to it, and I advise you not to use regExp on large chunck of text, that can have a significant impact on performances.
My old post on webscraping : http://myvisualdatabase.com/forum/viewtopic.php?id=1851
A good online regular expression tester : https://rubular.com
A STEP FURTHER
Most of the big websites like Goodreads, google books and so on, use REST API to deliver data to their pages and to third party application.
Using the REST API, you can request book data and get it from the server without all the html garbage you will have to deal with if you use html scraping.
REST API are just url that you query with parameters and that send data you in return, generally under xml or json format.
This is way faster and safer than webscraping because :
- the amount of data you receive is small
- data is organised in tree
- you don't have to worry about the web page format changing and ruining all your scraping procedures.
As an example of free API to get data here is what I tried :
https://restcountries.eu/rest/v2/name/france
Here, the parameter is the country name and I got in return data about my home country. (try it and see what data is displayed on screen after you followed the url - it is way shorter than a full web page).
All API works the same, and in book API cases, you pass author names or titles or ISBN numbers in the URL as parameters, and get your results sorted by relevance. (most of them request that you register to obtain a personnal token as and identifier, but it is free most of the time).
I strongly advise you look in this direction because you will save a lot of time and build a scraping system that will be independent from the web page format that can change without notice.
I don't know if Dmitry implement the JSON unit in MVD, but it is very easy to parse and get data from the json format with Delphi and, if not available yet in MVD, this could be a great addition.
Hope this helped a little
Cheers
Math
I'm a very good housekeeper !
Each time I get a divorce, I keep the house
Zaza Gabor