Saturday, December 15, 2012

The Urantia Book book indexing system

The platforms
Now that I have an embedded platform working on the Raspberry PI computer, I can now begin the conversion to different embedded systems. First Android smartphones, then Apple smartphones then Native Client for all computers. The indexing system does not use 3d so it can be the first to move to the Android platform and will be placed in the Android app store.

The need for a fast indexing system
First the program can be used to create indexing lookups for all the languages that are translated, irrespective of their fonts by using the current translations.

Next, I’ve found a need for a fast indexing system of the Urantia Book. So many times I’ve needed information fast but found the Internet not suitable when compared to native speeds. I have a working system on Linux and Windows but it has too many problems to be ultimately usable, the first is portability, the second is size. Speed is not an issue.

Also, I use this indexing system in the 3d Virtual Urantia trip to Paradise and would like it efficient with speed and size.

The Description of the program.
This is a description of a fast indexing and lookup system of the content of the Urantia Book. This is already working but it has a few problems. The method I use is instantaneous and works much like the Folio system which was once created for the book in the past. This following method is modernized with a GUI front end and has a coordinated thesaurus look up within the book. That is, a thesaurus that is unique to the Urantia book so that you can easily add multiple word lookups from the thesaurus window. For instance when you select home, abode shows up in the thesaurus window and can be easily added to the word list. This is complete and working in the desktop version.

I’ve found that the current databased index works well but is far too large and doubles the information in internal lists as it fills in those internal lists from the database (sqlite db). It takes a long time to download and it takes a long time to fill in these internal lists. So I skip all this wasted time (filling in the lists) and space (doubling the memory) problems by creating a hard programmed list internal to the program, which first, makes for easy transport as the book is internal to the program and it can be traversed mathematically instead of using indexed lists with positions and database indexing schemes. It has the added benefit of being as small and fast.

I have a lot of experiences with this work in publications at various positions I’ve held throughout the industry and I am familiar with the methods here presented.   

Most of the mathematics is done up front when creating the index and speeds the manipulations of the words at run time.

The method of creating the index:

The ‘words’ list.
First we create a list of all the words in the book in unique format. No word is repeated twice. This first list is be used when running the program (runtime).

Secondly, the program creates a list with the words and positions within the first list. This second list is used when creating the various following lists, that is, when we create the index (creationtime). It holds a hash based lookup system so that I can look up a word very rapidly when formatting the following lists.

The ‘book’ list
The ‘book‘ list is next. It is a list compacted together with 2 bytes per word in book order and these 2 bytes represent a position in the ‘words’ list. This saves a lot of space as saving the entire word takes one byte for each character of a word. Also this ordered ‘book’ list is used  for the ‘index’ and ‘content’ lists which hold a mathematic position within the book.

In this fashion we won’t need to save all the words in the book, it takes only two bytes per word. The recomposition speed in the runtime program is trivial using this method.

The ‘thesaurus’ list
The thesaurus list is formatted this way.
1. It’s position in the ‘words’ list. (2 bytes)
2. The number of words (1 byte)
2. followed by the index into the ‘words’ list of every synonym ending the synonyms for that word. (2 bytes for each synonym)

The ‘content’ list
The ‘content’ list holds all the title levels. It’s formatted this way:

1 The Title type. (1 byte)
2. The position in the ‘book’ list (3 bytes)
3. The number of words in the title. (1 byte)

The ‘index’ list
The index will be the biggest list and is formatted this way.
1. The index position within the ‘words’ list (2 bytes)
2. The number of occurrences within the book. (2 bytes)
3. 3 bytes for each word occurrence in the book. (3 bytes each)

The runtime program
My first issue is screen space. On desktops or tablets it’s not too much of a problem. There is a limited amount of space on the screen of a cell phone. Compounding this problem is the need for fast information exchanges.   

The program is separated into 6 virtual spaces each with their peculiar way of presenting the information.

1. The word index selection.
2. The references section
3. The Title content and paragraphs for quick perusal and scrolling.
4. The Title Contents of the book
5. The Book content proper.
6. Options

Breakdown of the 6 spaces.

The following comes from the working desktop version.

1. The word index selection.

The top left of the screen is the words to lookup in the book. Here you can place the words and operators. ‘home and father’ or ‘(home or abode) and father’ and the like. The bottom left is the a single word within the ‘thesaurus’ list. The right window is the ‘words’ list.

2. The References Section

Here we can scroll through each reference. When selected it jumps to the book content window where the book can be read from that position.

The paragraphs Section
Unfortunately I don’t have a screen shot at this time. I’ll add it later. This section is the same as the references section with the exception that the paragraph contents (words) are added. They can be scrolled. Clicking on a paragraph jumps to the book content window also.


















The Tree Section
The tree section is a very fast way to jump to any part of the book. By clicking on the tree leaves one can open that paper/section/subject very rapidly. When double clicked by finger or mouse,  the book content window is opened where the book can be read from that position.
 
The Book Section

This is where the book can be read. The word index selection is highlighted but can easily removed.
System Options Section.
Here are the different options for enlarging or shrinking the size of the text. I will add other options when the cell phone/tablet versions are ready.

Future thoughts
I will eventually add a comment section so that comments can be added to any paper/section/subject/paragraph/sentence of the book. Here I would like to add my internet programs to database the information for anyone who wants to make the comments available on the internet for anyone to peruse.
 



Thursday, November 8, 2012


I hope to get the system up and running in 2013.

Morten Sorvig I see, has been working on Native Client and Qt 5. Since Qt 5 looks like it'll be released in early 2013 then I should be able to get the Virtual Urantia system and the Urantia index and thesaurus up and running on Native Client then (At least that's the hope).

Also Qt has just released the news that they are folding the Necessitas project for Android into Qt and will be releasing a beta at the beginning of the year. IOS (apple smartphones), Digia promises will follow sometime after, although, they haven't locked down any dates.

Wednesday, August 1, 2012

If at first thou don't succeed, try and try again

I'm taking another run at getting QT up and running on Native Client. Sqlite and 3d aren't ready yet but I won't let that stand in the way this time around.

First I'll implement a small indexing program with QT using a database-less program and then with my main system I'll use a file as a database if I have to. I see there's movement on Sqlite and NACL so maybe I can help convert it if it's not ready soon.

Next is 3d. I see lots of companies have used it with NACL so if I can't get it working with QT then I'll make a stand alone 3 d module and communicate changes using JavaScript as a go between.

No matter what it takes ill make it work this time.

Friday, June 29, 2012

So close

On the last stretch now. I should have Internet capability by next week. Still have a long way to go before I have the platform I have in my dreams. Persistence and a willingness to never give up on  what I believe will make a difference. The Virtual Urantia Book is still just a cocoon waiting to take the step into a new age.Wishing though, doesn't do it. Only honest difficult, sincere striving to accomplish a dream and make it a reality.

Sunday, May 6, 2012

800 down 300 to go.

800 files converted. 300 to go. The brunt of the system is working. The brain is working within the browser. None of the graphic files are working yet. So a lot of work ahead. It's full steam ahead.

Thursday, May 3, 2012

Gaining a lot of traction on this Internet conversion.  550 files are converted out of 1100 files are converted to run in the browser. The 1100 files makes up the bulk of the code of the system.

There are thousands of issues to be resolved. Password databases and encryptions.  Also the logistics of placing it on the net so that all the peculiarities of dealing with local and remote file movements. Then the machine of updates.

At the very least, the trip to paradise will work and the index/thesaurus/viewer will be up and running for the book.

Hopefully 2012 will the year it makes it to the website.

Pierre 

Wednesday, April 25, 2012

NPAPI and artificial thoughts

Got most of the brain basics into the new Internet system and no major glitches. Now I'm moving the thought structures next. These next few weeks are exciting to say the least. It all works as naturally as can be. I work until my eyes start de-focusing  the computer screen. It's simply amazing.

I will go to Native Client when it's ready or at least be able to work on it and NPAPI for everyone that can't run it. That could be years from now. They will need to solve situations with file access and storage, P2P client communication and scores of other problems that have to be overcome for me to use it.


Pierre

Tuesday, April 17, 2012

It WORKS!!!!!!!!

Just got done testing 3d on my browser. Works well in Firefox and Chrome, Internet Explorer is next. Also, I have to do a separate compile on Linux, Windows, Apple, Android and all manner of cell phones and then up some up-front work so that the browsers know which one to send to your machine. Doesn't matter, my foot is in the door.

Years of waiting and now it's in my hand and it works. I am blown away.

Think I'll put the Urantia Book reader/index/search engine up first. It's tough to keep my excitement level down.

Pierre 

Monday, April 16, 2012

Lightning strikes with NPAPI

Wow. Knowledge is power? I've been complaining and lamenting that I couldn't use C++ on the Internet.  How could I have missed NPAPI. It's called a plugin and it allows you to run C++ code in the browser. It is ubiquitous. I've used it many times without knowing what it really was. A good example is a game called Quake. You can test it at www.quakelive.com. It is in full glorious 3d and it works really well.

And, it's been around since almost the beginning of the Internet. My frustration was completely unfounded. If I had just looked closer.........................

I thought I had to rewrite my program to use it in a plugin. It turns out that my software development kit has a plugin creator called QtBrowserPlugin and it allows me to put my system on the Internet. What??????????????????!!!!!!!!!

Free at last. Tonight I begin...................

Pierre


Thursday, February 23, 2012

Done creating the initial effects system that will power the viewer on the Internet using Native Client. It's time to begin work on getting all the objects from a 3d system to properly import into the 3d engine using the newer net-driven 3d OpenGL ES2 standard. The progress is really slow. I have my issues with understanding the mathematics of 3d world matrices. I should have paid more attention in math class. Anyway I can't imagine having the system being ready in 2012 so 2013 will be the year. The menu system is up to date and is being used by several people so that cell phone access will be ready when I launch.

Since I'm on the bleeding edge of technology, it's nice to know I'm not the only ones having problems with getting their system up and running on the net using C++. Most of the Native Client programs don't easily run on the latest Chrome browser versions. Native Client, I hope, will iron out most of it's difficulties and be ready for prime time sometime this year but I may be too optimistic.

Once again, as a developer, I wait for technology to catch up. The path I've chosen is sound and will work.

Thursday, February 16, 2012


Converting to OpenGLES2. The Internet won't work with regular OpenGL so it's time to convert and then I'll come back to Native Client. That will take me a couple months at least. Maybe QT5 with Native Client fully working will be ready for prime time. ????

It's pretty complex. The math is a bit daunting but not overwhelming. Someday, I'll have this system ready.  2013 maybe?

Pierre

Wednesday, January 25, 2012

No Joy yet

Back to the drawing  board. Since I can't Qt working on Native Client, I'm going to go back to what I know best until they've resolved their problems with getting Qt working on Native Client.

First thing up on the agenda is the trip to Paradise capability to start from planet earth, zoom out to the Solar System and then move in one clean swoop to Jerusem and do the same thing all the way to Paradise.

Here I'll kill cure two problems at once. First, the world of browsers only allow one window at a time for each program. So I'm creating a Master Scene that has the ability to create independent scenes and make them appear as if they we're part of the same scene. For instance: Let's say we have an Earth scene and we want to switch to the Solar System scene. With the right positions, re-sizing and transitions, bringing in the Solar System scene and removing the Earth scene can be done with the appearance that they are all the same scene. And this can be carried on all the way up to Paradise, making the trip a fluid trip from start to finish.

The same goes for automation. You can be viewing your entire factory or home and when you click on a certain machine, the 3d scene that animates and controls that machine can then appear in the factory scene without appearing to be a separate scene. In a home, you could switch to your washing machine or TV scene and make your changes and then on exiting, it would revert back to your home scene.

That will take me at least a month to get that working properly and it'll be fun every step of the way. Researching why something should work or not work when you're using someone else's program is not as much fun as creating it for yourself.

Saturday, January 7, 2012

Qt on Native Client


Qt on Native Client

Well I'm starting from scratch on Native Client so that I can get Qt running so that I can start changing my system. I've decided it is such an overwhelming effort that I would describe my musings on this jagged road.

The blog is at : http://naclqt.blogspot.com/ If you can stand the machinations of a mind in search of an answer at the core level of the difficulties of a developer like me then this is the place to see my meanderings.

Anyway I'm a week past my reflux and hyatal hernia surgery and I'm reinvigorated to get my system in front of the world.