Tuesday, November 29, 2011

CQWW CW - As the Dust Settles

Wow!  What a weekend - records falling in bunches, 5 bands open for contesting at once.  And I'm happy to report that the RBN was mostly up to the challenge.

First, the big numbers.  The RBN handled 1.578 million spots on Saturday, and 1.691 million on Sunday, or an average for the 48 hours of 18.9 spots/second.  This is roughly double last year's record average (also in CQWW CW), and is a measure both of how much the bands have improved and how many more people are contributing to the RBN.  Thank you all!

In case anyone wondered, we did have some trouble with the DX Spider Telnet server (telnet.reversebeacon.net, port 7000) on Sunday morning, as the load built to even a higher level than on Saturday.  Felipe PY1NB did some quick first-aid and got it running again within about a half-hour.  Meanwhile, the AR Cluster V6 server (arcluster.reversebeacon.net, port 7000) continued to deliver spots at full bore, though to a smaller audience than our main and long-established server.

There are also some signs that the load that CW Skimmer puts on Reverse Beacon participants' computers may be starting to cause problems.  A number of Skimmer ops reported trouble with less than 100% decoding of signals, due to excessive CPU loading.  At least the failure mode appeared to be graceful - my node, for example, stayed up unattended all weekend despite being on an anemic dual-core Pentium machine.

One surprise, at least to me, was the strong user demand for the main Reverse Beacon web page, which peaked at 384 simultaneous users, also on Sunday.  Log data suggest that most of these users were using the site to track spots of specific stations (maybe their own?), which puts an additional load on the database server.  However, the new hardware handled it very well, and that gives us a good level of confidence for the rest of the contest season.

Future plans?  Well, we intend to do some work on streamlining DXSpider so that it will handle the heavy throughput better.  There's no need for a lot of the features that put a drag on performance in the RBN server role - for example, the server doesn't accept DX spots from users, or Announce messages or WWV messages. Meanwhile, we're on the lookout for good new features to add to the mix.  Tell us what you'd like!

Thursday, November 24, 2011

URGENT

In a recent blog post explaining the features of the new Aggregator I asked operators of RBN Skimmers to set their Skimmer or Skimmer Server options (on the Telnet tab) so that Skimmer would pass all spots (not just CQ spots) to the Aggregator.  The intention of this was to enable spotting of beacons.  I also recently sent an e-mail to all Skimmer ops running less than current versions of the Aggregator, asking them to update.

Please ignore both of these requests, until after the high tide of contests this winterIf you have disabled "Post Only CQ Spots", please re-enable it. I had not considered how many people have lots of direct connections to their Skimmers' output, and if they disable the "Post Only CW Spots" option, those people will be getting all spots, not just CQs.  Moreover, just before CQWW CW and a flock of other major contests is a dumb time to try to update software.  Please leave well-enough alone!

Sunday, November 13, 2011

Opening Up SkimSrv to Get Everything in CQWW

With CQWW CW only 10 days away, and the RBN's new bigger server up and operational, we're just about ready to go.

However, as we all know, during big contests people will be running well outside the normal bounds of customary CW bands. Recently, we discovered that if you run Skimmer Server (SkimSrv) with the standard .ini file, even at 192 KHz bandwidth, you may miss some of the action. 

If you want to avoid such limitations (and you are running SkimSrv on a fast-enough computer), there is a way to make Skimmer copy CW out to the full bandwidth the QS1R receiver can handle. Be forewarned, though - we have no way of knowing how particular computer/SkimSrv/Aggregator/LAN/Internet configurations will handle the additional load. CQWW CW will, as usual, be the ultimate test, so we do not recommend implementing this unless you are going to be monitoring your SkimSrv computer during the contest.

First, you will need to be running Aggregator 1.4 and SkimSrv 1.3 - the latest releases.

the problem - the "stock" SkimSrv.ini file has the following line in it:

CwSegments=1800000-1890000,3500000-3600000,7000000-7070000,10100000-10130000,14000000-14120000,18068000-18095000,21000000-21100000,24890000-24920000,28000000-281500000,50000000-50110000

As you might guess, this instructs SkimSrv only to attempt to decode CW in the listed bands. Normally, this is a very good idea, to cut down on the number of decoders that might otherwise be spawned attempting to decode RTTY, PSK, etc. However, during a big contest, you'll miss a lot.

The fix is simple - just cut and paste the following line in place of the existing one:

CwSegments=1800000-2000000,3500000-3600000,7000000-7125000,10100000-10130000,14000000-14150000,18068000-18095000,21000000-21200000,24890000-24920000,28000000-28300000,50000000-50200000

When you do so, be sure that your e-mail program or text editor hasn't accidentally added any carriage return characters, because that will royally mess things up. Save it, and re-start SkimSrv.

A careful reader may have noticed that we've increased the CW segment on 10 meters to 28000-28300.  The reason for that is to allow you to set two center frequencies on 10, giving you the option of listening for the 10-meter beacons that operate between 28200 and 28300.  To do that, add a second 10-meter center frequency of 28209 to the CenterFreqs192 line in the .ini file.  With 7 bands, you will be able to cover the 6 contest bands plus the 10-meter beacon band.

I can't find my SkimSrv.ini file!


No problem. In Windows XP, it is in the SkimSrv folder of the Afreet folder, under Program Files. In Windows 7, it is found in C:\Documents and Settings\(user)\Application Data\Afreet\Products\SkimSrv, where "user" is the username you used to log on Windows.

What about non-contest periods? Do I have to remember to change back?

Nope. During non-contest periods, the RBN Server will set excluded frequencies to take care of that. Spots inside those bands - essentially, the normal RTTY segments - will not be sent to the RBN, just as if you still had the original CW segments line in place.

Help - it overwhelmed my Skimmer computer. What can I do, quickly?

Go into SkimSrv's Skimmer tab and reset your Skimmer bandwidth to 96 KHz. That will give you coverage of the first 90+ KHz of each CW band, and you will probably get 98 percent of the spots you'd get otherwise, but your computer loading should be usefully lower.  Alternatively, you can reduce the number of bands you are covering, and see if that is enough to get you by.

Thursday, November 10, 2011

Latest Improved Aggregator Released - 9 Nov 2011

Thanks to the programming skills of Dick, W3OA and Felipe, PY1NB, we now have a new production version of the Aggregator, the software that collects CW Skimmer or Skimmer Server spots and forwards them to the Reverse Beacon Network's server. This is much advanced over the beta versions that you may have been using (0.92b or 0.93b).
The new one (Version 1.4) is available for download here. It only takes about 5 minutes to get going. It offers several very significant new features, and we urge every RBN spot contributor to download and begin using it as soon as possible. If you encounter any bugs, please report them to w3oa@roadrunner.com; for operational questions, please write to me (n4zr@contesting.com).
In order to use the new Aggregator, you must be running the latest version of Skimmer or Skimmer Server (1.71 for Skimmer, 1.30 for SkimSrv). These are available free to registered users of either program, and can be downloaded from the DXAtlas web site.You must also be sure, if you are using CW Skimmer, to select "Allow SKIMMER commands" on the Telnet tab; SkimSrv allows these commands by default.
There is also one system requirement for the new Aggregator - you must have Microsoft .NET 4.0 installed on your computer. If you were running version .92b or .93b of Aggregator, or a growing number of other recent software applications, you already have this on your computer. You can get it here, and it takes only 5 minutes to download (on broadband) and install. This is the reason why the Aggregator is only 90 KB - and since it is increasingly a requirement for other software as well, we hope this won't be an obstacle for anyone.
All you need to do is save the new Aggregator somewhere, and it is ready to go. For users of Windows Vista and Windows 7, you will need to put the new Aggregator somewhere where the operating system will allow it to write an .ini file. That means putting it almost anywhere but the Program Files directory - many of us use C:\Ham Radio as an alternate location, to avoid these issues.

Some security software (Norton Security's SONAR, for one) will try to block Aggregator from running, and even removing it from the installed location. These packages make their "decisions" based on what they regard as "risky software behavior." We can't really avoid getting them stirred up, given the way that Aggregator operates. If you run into this, you will need to tell the busybodies that you know best; all these security software packages have provision for reversing their "decisions." With SONAR, you get to it through the “Details” link, and then by “Options” – you can tell it to ignore the program from then on.

Here is a screenshot of the new Aggregator - probably the best way to explain its features.
The biggest thing you'll notice is that the new Aggregator is a "real" Windows program, which means that you do all the setup on a single screen, rather than having to set command-line options. Despite that, it is very small, uses essentially no CPU power, makes no changes to the registry, and does not have to be installed.

The upper panel is the initial setup

The default settings should normally work for your installation. If you have not set up your Skimmer or SkimSrv to require a username or password for access to its Telnet server, then you don’t need to fill in either one. The IP address is the address of the computer Skimmer or SkimSrv is on. If Aggregator is on the same computer, then the correct fill-in is 127.0.0.1. Otherwise, fill in the internal IP address of the Skimmer/SkimSrv computer. The port number must correspond to the one you have set on the Telnet tab of Skimmer or SkimSrv.
If you lose your connection to Skimmer for any reason, the green message at the bottom of the pane will change to red to alert you.

The next pane down controls what spots are sent to the Reverse Beacon Network. This is where things get really interesting.

The left-hand column is pretty self-explanatory, except for the last item. The “Add a base frequency” section is intended only for transverter users. Do not attempt to use it to correct your frequency calibration; instead, refer to this entry in the RBN blog for a quick and easy way to calibrate your receiver within 50 Hz.
In the right-hand column, the new Aggregator provides the functionality of the "Paranoid" validation setting in CW Skimmer, which some users of SkimSrv may have been missing. If this option is selected, only spots that match calls in a master file are forwarded to the RBN. The Aggregator uses the text version of the widely-used master.dta file, called master.scp. To download it, click the button in the SuperCheckPartial pane in the right column of the Aggregator.
“Paranoid” validation is a two-edged sword. Because the latest master file is only compiled before major contests, it is virtually certain to omit new special calls and DXpeditions. You’ll have to decide for yourself whether you want to exclude them from the spots you send. Experience indicates that either Normal or Aggressive validation, set in your Skimmer or SkimSrv, provides quite adequate protection against busted spots.
At the bottom of the pane you’ll see confirmation that you’re connected to the RBN. Also included, and new in this version, the server can request only certain spots, and exclude bands of frequencies. As shown here, beacon spots will now be forwarded to the RBN, and spots in certain frequency bands will not. The idea is to avoid busted spots by excluding the main RTTY frequencies on the higher bands. This is a bit of an experiment, because in major contests the boundaries are often ignored. For that reason, any excluded frequency bands will likely be dropped from the server’s instructions during contests. You’ll note that in the full-screen image above there are no excluded frequencies shown

Once you are using the new Aggregator, please uncheck the "Post Only CQ Spots" option on the Telnet tab of your Skimmer or SkimSrv. That will send everything to the Aggregator, and it will decide which spots to forward. This will facilitate experimentation with new features, while not loading the RBN server with unwanted spots, and it’s the only way beacon spots can get to the RBN.

Again, the green text at the bottom of the pane tells you that all is well, and changes if there is something that needs to be corrected.

Moving on to the bottom left pane …



This is an added feature that some users will find useful. It lets you feed spots locally from the Aggregator to a logging program, and choose whether you want to see everything locally or only spots that have passed through the Aggregator's filters and been sent to the RBN. Again, the text at the bottom tells you the status of this feature - when a local "user" is connected, the text changes to green and reports that the connection has been made.

The top right pane of the Aggregator window is another new feature, to be used for messages from the server to Skimmer ops. It might be used, for example, to advise that a new Aggregator has been released, or a new blog article, or to provide information to help correct some issue that has arisen. It will be worth checking here periodically to see if there’s anything new.

The next pane tells you whether your master.scp file is the same one available from the RBN server, and provides a Download button that is greyed out unless a newer one is available.

And finally, the last pane.
It gives you all the information collected on each spot. Note, though, that it displays the spots as they are provided by Aggregator to the local Telnet port. In this case, I have SkimSrv sending all spots to the Aggregator, be they CQ, DE, or undetermined. The program will take care of sending only those spots that the server wants.

So that's the story. We hope you will all thank Dick and Felipe for their contributions, by downloading and using the new Aggregator.

73, Pete N4ZR