Alice Developer's Journal Archive
These are the earlier entries to the journal. You can find the recent entries
on the main journal page.
Confused about these weird dates we use? Check our Frequently
Asked Questions for an explanation.
2000:318
All the features i mentioned yesterday are in place. The one caveat is regarding
the defaulting of the UBR window to the All Cards overview when a new UBR is
displayed. This only happens when selecting a new UBR to view by clicking on
its meter in the Network window. Using the controls on the UBR window to change
the displayed UBR do not switch the displayed card. This was intentional, and
can be modified if desired.
I haven't met with Colin yet but Trevor says that Alice is seeing good use
in the call centre, so i've decided to begin the alerts at my leisure. But first
i'll be taking some time off, i don't know when i'll get back to work.
2000:317
Trevor tested the new build yesterday, and sent me some screenshots of the
new meter styles. Here's Alice under field conditions showing relative
and absolute style meters. Note how the relative
view shows that BUSY2 and SACO are down by 5 and 10 percent respectively,
but the absolute view shows that the actual number of offline modems do not
constitute a significant failure. Each style of meter gives a complementary
view of the data, and together they give a more complete picture of the state
of the UBR than either one alone could.
Also, through field testing Trevor was able to identify a number of interface
issues that could be improved, and i'm in the process of implementing them now.
Among the planned changes are:
- Making the UBR window default to the All Cards overview when a new UBR is
selected.
- Tracking the number of queries generated automatically and by the user.
- Indicating a UBR's monitoring status on both the UBR window and the Network
window.
- Splitting the UBR meters on the Network window into 2 rows to accomodate
more UBRs.
2000:315
Added a mini-feature, meter dimming for UBRs whose queries fail. The program
accomodates 3 failure states (server timeout, requested document not returned,
and document contains no data). Any of these cause the meters for the corresponding
UBR to be dimmed (both in the Network window and the UBR window), informing
the user that a query has failed but still allowing them to see the old data.
The "Time since last update" panel on the UBR window status bar shows
the time since the last successful query (and thus the age of the data being
viewed), but the period between query attempts remains at the specified Query
Interval. At present the program does not identify which of the 3 failures occurred,
but it would be simple to make it do so if such information would be useful.
2000:313
Posted another minor update. Tom suggested that on the UBR window, the user
should be able to type in a value for the query interval as well as use the
up-down control to make adjustments. I left this feature out originally because
i foresaw that it would be a non-trivial exercise, and indeed it took quite
some time to make the algorithm robust. To get an idea of the complexity,
you can look at the final flowchart. I had to
write an extended version of the InStr function to accomodate searching for
patterns, which is done in the first 2 decisions in the flowchart. There's
actually more code involved in handling the query interval textbox than there
is in handling the entire query queue!
We're still waiting on Trevor and Colin, so it looks like regular development
may not be resumed this week. In the meantime i'll likely continue to tweak
the details, but i won't work on any major new features.
2000:311
Posted a very minor update to Alice. I added meter ticks to the absolute style
meters and changed the ones on the relative style meters to be consistent.
Further development of Alice is probably going to be suspended for a while.
I want to give Trevor a chance to do some solid production testing, plus i'm
interested in getting word from Colin. Hopefully we'll know by Tuesday how things
stand, which will determine when the next round of features (namely the alerts)
can be implemented.
2000:310
I've posted a build. The following features have been added: individual UBR
queries, adjustable per-UBR query intervals, choice between relative and absolute
meter styles. There are two caveats.
The first is regarding the choosing of meter styles. There are two grouped
buttons on the Network window, one for each style. There are also two identical
buttons on the UBR window. Both of these sets of buttons have a global effect,
they have been placed on both windows only for convenience. Changing the selection
on one window should update the button states on the other window, and for the
most part this is true. However when the program first starts, making a selection
on one window does not correctly update the button states on the other. If you
make a selection on both windows, thereafter the buttons will be in sync. I
have tried everything i can think of to get around this, but it seems to be
a bug in the toolbar control. It is only an annoyance, full functionality is
still present.
The second caveat is that occasionally during program termination, an illegal
memory reference exception will occur. It doesn't happen often, and is unpredictable.
I suspect it has something to do with the state of the query queue upon exit,
but i haven't been able to investigate it yet. Again, this is only an annoyance;
since the error occurs at program termination the impact is negligable.
Other than those two small details, the program operates quite smoothly. The
object model and code, too, have been smoothed out considerably. The stage is
now set to implement the alerts.
2000:308
The NetworkMonitor class and its query queue are slowly taking shape, it would
have been much quicker if i'd planned all this out before i began. This certainly
illustrates the value of following a design methodology. I installed Visio,
and have been investigating it as well as Visual Modeler. I think UML (Unified
Modelling Language) is something i'm definitely going to have to learn, as it's
the industry standard OO design methodology. Anyway, for now i'll just have
to carry on with what i've got.
All the querying code has been moved from frmNetwork into NetworkMonitorClass.
I've added an "Update" button to the UBR window, and the queuing is
now operational. Next i've got to let each UBR have its own query interval,
which also should be adjustable on the UBR window. Once that's in place i think
i'll be ready to post a build.
2000:306
I had a meeting with Trevor last night. I found out that Alice was quite a hit
among the TSAs. Also, Colin from the NMC saw the program and is going to talk
to the NMC chief about possibly picking it up. It looks like there may be a
possibility for some money out of this after all. The problem is that there's
a lot of political posturing and bureaucracy going on, and if we really want
to get Alice in we'd have to play their game. We've decided we're not interested
in doing that, instead we'll continue to develop Alice solely for Trevor's use.
If Cogeco wants to pick it up, great; if not, it'll be their loss.
During the meeting we identified the features to be implemented for the next
round. These will include the ability to query UBRs individually, multiple meter
view options (relative or absolute scale, display of only selected states, i.e.
online, ranging, offline), and the alerts. The first priority will be individual
UBR queries. The need for this was demonstrated during the test; when a particular
UBR experiences a failure, the TSA will want to monitor that UBR more frequently
than the others.
To implement this feature i'll have to use a query queue. As each UBR's data
expire, it will be added to the end of the queue. If the user initiates a query
on a UBR, it will be added to the front of the queue. The queue will be continuously
processed as long as there are entries in the queue. This architecture is perfectly
suited to the NetworkMonitor class being planned for the client/server version
of Alice.
2000:305
We had the first live test of Alice beta today. The first time it was run,
all the UBRs were being queried but no numbers were being returned. A quick
test using HTTPTest revealed that there shouldn't be a question mark preceding
the POST data, as i suspected. I removed it from Alice and thereafter it worked
flawlessly. Trevor said they'd added a new UBR between the time he tried the
alpha and the beta, so we added an entry to the INI file and Alice was able
to recognize and query the new UBR correctly. Then, just as Trevor was comparing
the numbers returned by Alice and those by LG, an outage occurred on two UBRs.
He was able to see it immediately on Alice! The first live demonstration of
the beta could not have gone better. We took two screenshots of Alice, before
and after the outage. I'm extremely pleased
with what we have achieved thus far.
Tomorrow i'll begin work on the next round of features. I'll post a complete
description of my plans then.
2000:303
It's 1:17 right now and i haven't posted Alice yet... but i have good news,
i solved the problem! The one i mentioned a couple days ago about HTTPTest not
being able to get a proper result out of my fake LG. It's now working 100%.
Again, it was all in the headers... i started reading "Web Client Programming
with Perl", and in a matter of minutes i had the solution. I wasn't specifying
the content type of the POST data! Apparently Apache isn't so demanding, but
IIS can't seem to do without that information. That explains why it worked at
the office when Steve tried it, but it didn't work here. I should have turned
to that book first, after all it was what gave me the necessary knowledge of
HTTP headers to do authorization. Anyway, this means a new version of HTTPTest
needs to be posted.
As for Alice, the reason i haven't posted it is that Trevor won't be able to
test it until Monday. I've decided to take this time to tweak some of the interface
features. Instead of showing the time of the last update, Alice now shows the
time since the last update, since that's really what the user is interested
in. It was easy to do, i just took advantage of the timer which is already in
place for automatic updating. The "Update" button on the toolbar is
now disabled when an update is in progress. I plan to add a button to the UBR
window which will query that UBR only; this should be especially useful if the
query interval is large. I'll need to show the time since the last update for
the currently displayed UBR as well.
You can expect Alice beta to be posted sometime tomorrow (that is to say, later
today). I'm still holding off on the alerts until i get good results from field
testing.
2000:302
Alice has achieved beta status. Direct LG querying is in place, and the option
of automatic updates is available too. If the QueryInterval key in the INI file
is 0 then no auto-updates will occur, else it will occur although the interval
is bounded to a minimum of 300 seconds. I just have to tie up a few loose ends
and i'll post the beta on the download page.
It looks like Tom won't be able to assist me with Alice anytime soon, as he's
tied up with his own research at present. However he has expressed interest
in taking over development of HAND with a view to releasing it for public use.
He's already come up with a name for it: Octopus. It'll be a while before we
can give it our attention, but it's on the plate.
2000:301
Ran into a major problem today. The fake LG i set up doesn't seem to respond
correctly to queries submitted by HTTPTest, although it does respond to the
HTML form. This may not be a serious problem, because we have confirmation from
Steve that HTTPTest works with the real LG, but without an operational fake
LG here, it'll be difficult for me to test things.
Specifically, the problem is that when i do a POST request from HTTPTest, the
ASP doesn't seem to recognize the post data, although it is reporting the correct
number of bytes sent. It is definitely receiving the post data, but somehow
is unable to parse it the way it does with queries from the HTML form.
As a quick fix, i could take out any parameter checking and have the ASP always
return a modem summary, which would be enough to let me get my work done. Still,
it's puzzling that it's not behaving as expected, and i really want to find
out why. Alice takes first priority though--must get a beta to Trevor as soon
as possible!
Later that day...
Well, i just couldn't resist. I had to find out why the ASP wasn't cooperating
with HTTPTest, so i installed InterDev hoping that it would give me some insights.
Unfortunately the MSDN Library help for InterDev appears to be in shambles--over
half the topics i looked up referenced missing pages! Unforgivable.
The frustration resulting from that experience prompted me to modify the ASP
as originally planned and get on with it. *Sigh*...
2000:300
Most of the work for directy querying is done. I disabled the automatic updates,
i'll enable it again after i know the direct querying works properly. The only
thing left is putting all the pieces together, i should have it complete by
tomorrow, possibly including the auto-update if Trevor can test it the same
day. However the alerts won't make the first cut of the beta, since it involves
adding new interface elements.
Tom and i fixed another small bug in HAND and found a few others. We have a
vague idea of possibly beefing up the program... not sure what we'll do with
it yet.
2000:299
Things are going somewhat slow in putting in the direct LG queries. There's
no actualy delay, i just find myself cleaning up a lot of the peripheral code
at the moment... i want to have the object model fairly robust before i add
direct queries. I'm looking ahead to implementing the client/server version
of Alice, and want to make that sure that process goes as smooth as possible
by laying the foundation now.
I've moved all the assorted configuration settings into a single Config object,
which has methods to load and save settings via the INI file. Next the UBR class
is going to get a thorough renovation, and then i'll put in the direct queries
(code for which i wrote and tested separately). I expect i'll post a build after
that, before starting work on the alerts.
2000:298
Made some small refinements to the fake LG script. I'm now writing the code
to query the script directly and extract the modem summary information.
There was a bug in the modified HTTPTest (which i'm calling "Hand").
Any files that were downloaded had 6 extra bytes appended to the file. The bug
was easy to correct, but i also wanted to repair the corrupted files rather
than download them again. I threw together a little command line utility in
C++ to do the job. It took me a couple hours, i've really let myself get out
of practice. Used to be i could write something that simple in a matter of minutes.
At least i'm getting back into it now.
2000:296
I've put up a fake LG on my server using ASP, so i have something to test
Alice against. However, once i did that i got distracted with modifying HTTPTest
to allow the user to see all the links on a page and to select and download
a subset of them. I've got it pretty much working, but without any bulletproofing,
since i expect it only to be used in-house. Anyway, now that that's out of the
way, i can begin the task of adapting Alice to query LG direct. The next task
will be putting in the alerts, and to facilitate that i've already made the
fake LG change over time by returning random results on each query.
Once those two features are in place, i'll bump Alice up to beta and let Trevor
put it onto production testing. From there we'll begin work on the client/server
version of Alice.
2000:294
I've been researching ActiveX servers. I think a good way to proced would be
splitting Alice into 2 components, a server which does all the monitoring and
maintains the network object, and clients which connect to the network object
on the server and retrieve the current status. The clients can query LG themselves
when checking individual modems, and talk to the server when checking the status
of the network. Also this would facilitate Trevor's idea of requiring clients
to log onto the server, so it can track who does what when an alert event happens.
I'm going to bring Tom aboard to assist with the project; i think we'll be able
to manage it.
As for the features planned yesterday, i'm going to drop scaling the meters.
There just doesn't seem to be any advantage to showing the abolute number of
modems on a UBR. In the same way that a window for each UBR was unnecessary
because users wouldn't typically want to compare UBRs, so too is it unnecessary
to visually indicate the number of modems on a given UBR relative to others.
That information is available in the tooltips, and i think that will suffice.
2000:293
I've been thinking about the features mentioned in the previous entry, about
their desirability and what would be required to implement them. I'm becoming
increasingly convinced that i should start from scratch, re-architect the whole
program. The reason is that up until this point, Alice has pretty much been
just a hack, a little something whipped up to make the TSAs' jobs easier. But
considering the leaps and bounds we've made over the past few days, i think
Alice could become a serious suite of network diagnostic tools. To do it properly
necessitates creating a robust object model and a client-server architecture.
A challenge, to be sure, since i've never done anything like it before. As a
side benefit, restarting would force Alice to earn the 2.0 designation! I'll
have to consult with my associates and formulate a solid plan before proceeding.
2000:292
There was a little trouble in faking the Referer header to LG, which was resolved
fairly easily, (once again) thanks to Trev's magic. He tried the HTTP test program
at the office, and found that the authorization worked but LG was still balking
at the Referer header. My only idea was to get a packet sniffer or port spy
to see exactly what was going on, but when Trevor came over tonight he realized
that i had overlooked the difference between GET and POST requests. The test
program was only making GET requests, so i modified it to allow POST as well.
We sent it to Steve who was on overnight at the office, and he was able to get
perfect results out of LG. At last, the old bugaboo is dead. This victory has
renewed our interest in the possibilities of Alice, and we've decided to continue
developing it for its own sake. We've come up with a number of interesting ideas,
but for the immediate present the ones i'm going to implement are:
- Get Alice to query LG direct (of course).
- Scale the UBR meters relative to how many modems they contain.
- Alerts when a node drops below a certain percentage of modems online.
2000:291
Eureka! We now have a way for Alice to retrieve documents that require authorization.
It was a little more involved than simply using the UserName and Password properties
of the Inet control. It turns out that those properties only apply to FTP transfers
(of course that significant little tidbit wasn't mentioned in MSDN Library).
When i discovered that, i almost gave up on the Inet control, thinking that
i'd have to code straight to sockets. So, resigning myself, i began digging
into socket programming and the mechanics of HTTP transfers. But, as fortune
would have it, HTTP authentication is done entirely through the headers, and
the Inet control does in fact provide control over the headers--not total control,
but enough to get the job done. I consulted Trevor, and after he'd worked his
magic, we had the problem solved. The key was understanding the format of authorization
headers (in particular, BASE-64 encoding). And as a bonus, now that we understand
headers, we can fake the Referer header and get around the LG script limitation.
Alice has been unleashed!
The issue that concerns us now is what exactly we want to do with this new
found power. At the very least we can upgrade Alice to bypass the FileDog nonsense
and talk to LG directly. Beyond that, we could generalize Alice to incorporate
other tasks that require communication with servers. But i have to think about
how much work i want to do on it, especially since the likelihood of monetary
recompense is pretty much nil.
2000:290
The status bar on the UBR window is now operational. The architecture has been
improved somewhat, but not as i had intended. My idea was to put a reference
to a meter in the UBR class, which would allow the UBR objects to update their
corresponding meters directly. To my surprise i discovered that when using the
reference, only extrinsic properties and methods were available! I had to leave
the UBR class accessing its controls directly through their parent form as it
was. Strange behaviour. C++ wouldn't tolerate such things.
I revisited the old Internet Transfer Control in the hopes of circumventing
the authorization problem. To summarize, the original problem started when Looking
Glass became password-protected. I thought Inet couldn't handle passwords on
HTTP requests, but Trevor pointed out that we could simply specify the username
and password inline in the URL. I think it would have worked, had the password
not included the '@' character. That basically put Alice on ice, until they
started using FileDog to retrieve the modem summary report and save it as a
text file. Alice now reads that text file, but for a number of reasons we'd
prefer Alice to fetch the report itself. I'm not sure what motivated me to look
at the control again, but it paid off. I discovered the UserName and Password
properties (don't know how i missed them before), which can be set to any string
value. I don't know if it works, the necessary experiment hasn't been conducted
yet (i don't have authorization on any secured sites). I wrote a small program
to allow such testing, and expect to get some results from Trevor soon.
2000:289
This is the first entry since i've started keeping track of updates to Alice.
The most recent development is that the automatic update timer has been implemented.
There's now a key in the INI file called UPDATE_INTERVAL which will be the time
in seconds between retrieval and processing of the modem summary report. The
granularity of the timer has been set to 5 seconds to reduce processing overhead.
With this addition the program is usable feature-wise. I expect to have Beta
status shortly.
Previous to that the major change was recoding of the meters and readouts to
handle ranging modems. Also, the interpretation of the compact modem summary
format [ 12 : 34 / 56 ] has been changed so that the second number now means
ranging modems instead of online modems. Originally i had intended to add ranging
while leaving online in place, but i felt having 4 fields would be cumbersome
and in any case one doesn't really care about the online modems as much as the
misbehaving ones.
A minor detail, i finally figured out why the display of the splash screen
was so choppy. The intent was to have the initialization take place while the
splash screen is up, but what has happening is that initialization took place
even before the splash screen was painted. Calling Show on the form sends a
message to the window to repaint, but it wouldn't be received until other tasks
had completed. This problem was resolved by directly calling Refresh on the
form before proceeding.
One important thing that needs to be added is the status bar updates on the
UBR window. This was removed mainly because under the new architecture it wasn't
clear where that code should most naturally be placed. Actually this is a weakness
of the new architecture, and to adequately resolve it i'll need to compartmentalize
the objects further.
| 
