When we'll see elinks for MSX?

Pagina 5/7
1 | 2 | 3 | 4 | | 6 | 7

Van hit9918

Prophet (2882)

afbeelding van hit9918

17-11-2018, 18:13

where is the actual work of a browser.
what is needed is a LAYOUT ENGINE!

you might say "but I just simply want a text browser".

but. imagine a text editor with word wrap.
the line numbers on screen dont match the line numbers in the file when you got word wrap.
so how does one display this stuff onscreen?
one got to make string snippets that are max as wide as the screen... a list of snippets... snippet object lists...

a long story short, a text browser needs 95% of effort of a graphics browser!

text browsers were made for unix situations that got only a text console
the reason was NOT that a text browser would be so much more easy than graphics
and the reason was NOT cpu speed

Van Grauw

Ascended (8612)

afbeelding van Grauw

17-11-2018, 21:23

A text browser is easier in that for a first step you can restrict yourself to plain HTML processing and forget about CSS or graphics mode access and performance, fonts etc., and can focus on parsing the input and setting up the lazy processing pipeline.

Focusing on solving the problem a step at a time, with already-usable intermediate results, rather than taking it on as one gigantic project as a whole. Then add features step by step.

In the first step I would read .htm files and output mildly formatted text to console (just a viewer). Next step, add UNAPI support to read data via HTTP. Next step, speed up the text output with direct VDP access. Next step, word wrapping. Next step, link highlighting and traversal. Next step, process some more difficult markup like tables. Next step, process CSS files to add support for some simple styles like text alignment. Next step, switch to a graphics mode with proportional fonts. Next step, images. Next step, bookmarks. Next step, more CSS. Etc.

From step one you have something useful, and the first iterations might even work on MSX1. If you quit development at any time, the community will still have something usable, and you don’t overload your brains with a massively complex body of work you need to deal with at once. This could be a project with the potential of a huge scope, so I would tackle it in this way. (As you may notice, VGMPlay also doesn’t have a UI yet. Wink It’s the same development philosophy.)

Van edoz

Prophet (2189)

afbeelding van edoz

19-11-2018, 10:51

Grauw wrote:

In the first step I would read .htm files and output mildly formatted text to console (just a viewer). Next step, add UNAPI support to read data via HTTP. Next step, speed up the text output with direct VDP access. Next step, word wrapping. Next step, link highlighting and traversal. Next step, process some more difficult markup like tables. Next step, process CSS files to add support for some simple styles like text alignment. Next step, switch to a graphics mode with proportional fonts. Next step, images. Next step, bookmarks. Next step, more CSS. Etc.

Yes it would be a good idea to do it step by step. I the end it would be a huge step forward for the MSX system in general Big smile Big smile Big smile and .. indeed.. browse the internet using a MSX 1 will be crazy : D:

Van edoz

Prophet (2189)

afbeelding van edoz

19-11-2018, 11:36

BTW. a good example is "links browser" This one have a text only browser and a graphical one. I just installed it but i works well for basic browsing.

Van skumlerud

Resident (46)

afbeelding van skumlerud

19-11-2018, 12:23

And the links binary is somewhere around 5Mb in size. Source code is ~7Mb compressed, and that's without all the supporting libraries. And this is the simplest browser around.
You can also take a look at Netsurf which is an extremely lightweight browser originally developed for RiscOS. The Atari port is barely usable (slow!) even on a 250Mhz Firebee. On an MSX you wouldn't even have enough RAM just to load the frontpage HTML for the majority of current websites, and this will get even worse.
As said earlier, the only way I can imagine usable web browsing - even in text mode only - on an MSX is to go via a proxy that does everything for you.

Van Grauw

Ascended (8612)

afbeelding van Grauw

19-11-2018, 13:30

You can’t compare a *nix binary to MSX.

Based on a technical assessment what it needs to do, I see no problems for this being possible on Z80. (And given my work on VGMPlay and gunzip, which process large amounts of data up to several MBs, I do have some experience dealing with such large files to back up this assessment Smile.)

Executable code for a basic text browser can fit in 32K I think, and for the document tree how much memory you need depends on the page size (especially the amount of text content), but I think 512K memory should get you far enough to browse most pages.

There is absolutely no reason I can see why you would need hundreds of MHz for this kind of task, even at 3.58 MHz as I said you should be able to present the top of the content within a second. In the end it is just read some data, parse some markup, and present it to the user, done.

Note that the lazy processing is key for good memory usage and performance; for comparison, lazy processing is also high on my wishlist for VGMPlay MSX to reduce the VGZ decompression time. If a 3 minute song takes 30 seconds to decompress, currently you have to wait 30 seconds for it to play. But during playback there is enough idle time left to do decompression, so why not do it on the fly. Then it could actually play sooner than uncompressed files, because less data needs to be loaded from disk.

What’s holding me back there is my efficient decompression implementation means I have to implement some form of coroutine (primitive cooperative multitasking) and memory context switching, whilst not impacting playback timing, which is a bit complicated. I think a HTML processor can be set up more simply, especially when it’s built from the get-go with lazy processing in mind.

I really want to make a prototype to prove this, but I’ve got so many projects already. I hope fr3nd will pick this up as a project Smile.

Van AxelStone

Prophet (2723)

afbeelding van AxelStone

19-11-2018, 14:24

Grauw wrote:

You can’t compare a *nix binary to MSX.

In fact a basic version of Unix itself is available for MSX without any RAM expansion, I'm talking about Uzix, which as I mencioned before has even a visual browser (FudeBrowser).

Van fr3nd

Expert (92)

afbeelding van fr3nd

19-11-2018, 19:45

Quote:

I really want to make a prototype to prove this, but I’ve got so many projects already. I hope fr3nd will pick this up as a project.

I'd very much like to to do it. I've actually started investigating how to implement it but I realised it's way more complicated than I expected.

I already have implemented most of the http communication in MSXHub, so this part would be easy.

I think the html parsing should be possible too. I've found a tutorial called "Let's build a browser engine!" which shows you how to create a very simple engine capable of parsing simple html (and even css!). It's in Rust, but examples in C and other languages can be found easily.

But... the biggest problem I see here is the memory. To implement this so called "simple" parser, the whole DOM needs to be stored in memory and nowadays most of the websites (just the html, no images, no css) are quite big:

I still find the concept of mapped memory confusing and I'm not even sure if it's possible to use it using SDCC. For me, assembler is (at the moment) too complicated for my current knowledge...

So I'd need to find a way to parse the html without storing the whole the DOM in memory... I'll keep investigating.

Van edoz

Prophet (2189)

afbeelding van edoz

19-11-2018, 20:20

Yes, it is not easy because the web is very dynamic now days. And yes, the size for pages are very big. I think your examples are already small. Most websites are above this 164 k i have seen up to 2 mb.. Wink So processing will be not easy.
You should maybe store the file on disk and read parts of it. I think the HTTP downloader (wget) will be the most easy part Big smile

If i remember correctly DOX supports this streaming technology. Which allows you to read a file from top down and show already the content what was loaded. But this is used for SymZilla in SymbOS and probably can not be used in this case. But probably you need to parse HTML, show it and store the rest on disk or so.

Van skumlerud

Resident (46)

afbeelding van skumlerud

19-11-2018, 22:07

AxelStone wrote:

In fact a basic version of Unix itself is available for MSX without any RAM expansion, I'm talking about Uzix, which as I mencioned before has even a visual browser (FudeBrowser).

FudeBrowser does (almost) exactly what I've suggested a couple of times in this thread - it does not parse the web-pages itself but offloads almost everything to a server running on a PC.

Pagina 5/7
1 | 2 | 3 | 4 | | 6 | 7