Latest Improved Aggregator Released - 9 Nov 2011

Thanks to the programming skills of Dick, W3OA and Felipe, PY1NB, we now have a new production version of the Aggregator, the software that collects CW Skimmer or Skimmer Server spots and forwards them to the Reverse Beacon Network's server. This is much advanced over the beta versions that you may have been using (0.92b or 0.93b).

The new one (Version 1.4) is available for download here. It only takes about 5 minutes to get going. It offers several very significant new features, and we urge every RBN spot contributor to download and begin using it as soon as possible. If you encounter any bugs, please report them to w3oa@roadrunner.com; for operational questions, please write to me (n4zr@contesting.com).

In order to use the new Aggregator, you must be running the latest version of Skimmer or Skimmer Server (1.71 for Skimmer, 1.30 for SkimSrv). These are available free to registered users of either program, and can be downloaded from the DXAtlas web site.You must also be sure, if you are using CW Skimmer, to select "Allow SKIMMER commands" on the Telnet tab; SkimSrv allows these commands by default.

There is also one system requirement for the new Aggregator - you must have Microsoft .NET 4.0 installed on your computer. If you were running version .92b or .93b of Aggregator, or a growing number of other recent software applications, you already have this on your computer. You can get it here, and it takes only 5 minutes to download (on broadband) and install. This is the reason why the Aggregator is only 90 KB - and since it is increasingly a requirement for other software as well, we hope this won't be an obstacle for anyone.

All you need to do is save the new Aggregator somewhere, and it is ready to go. For users of Windows Vista and Windows 7, you will need to put the new Aggregator somewhere where the operating system will allow it to write an .ini file. That means putting it almost anywhere but the Program Files directory - many of us use C:\Ham Radio as an alternate location, to avoid these issues.

Some security software (Norton Security's SONAR, for one) will try to block Aggregator from running, and even removing it from the installed location. These packages make their "decisions" based on what they regard as "risky software behavior." We can't really avoid getting them stirred up, given the way that Aggregator operates. If you run into this, you will need to tell the busybodies that you know best; all these security software packages have provision for reversing their "decisions." With SONAR, you get to it through the “Details” link, and then by “Options” – you can tell it to ignore the program from then on.

Here is a screenshot of the new Aggregator - probably the best way to explain its features.
The biggest thing you'll notice is that the new Aggregator is a "real" Windows program, which means that you do all the setup on a single screen, rather than having to set command-line options. Despite that, it is very small, uses essentially no CPU power, makes no changes to the registry, and does not have to be installed.

The upper panel is the initial setup

The default settings should normally work for your installation. If you have not set up your Skimmer or SkimSrv to require a username or password for access to its Telnet server, then you don’t need to fill in either one. The IP address is the address of the computer Skimmer or SkimSrv is on. If Aggregator is on the same computer, then the correct fill-in is 127.0.0.1. Otherwise, fill in the internal IP address of the Skimmer/SkimSrv computer. The port number must correspond to the one you have set on the Telnet tab of Skimmer or SkimSrv.

If you lose your connection to Skimmer for any reason, the green message at the bottom of the pane will change to red to alert you.

The next pane down controls what spots are sent to the Reverse Beacon Network. This is where things get really interesting.

The left-hand column is pretty self-explanatory, except for the last item. The “Add a base frequency” section is intended only for transverter users. Do not attempt to use it to correct your frequency calibration; instead, refer to this entry in the RBN blog for a quick and easy way to calibrate your receiver within 50 Hz.

In the right-hand column, the new Aggregator provides the functionality of the "Paranoid" validation setting in CW Skimmer, which some users of SkimSrv may have been missing. If this option is selected, only spots that match calls in a master file are forwarded to the RBN. The Aggregator uses the text version of the widely-used master.dta file, called master.scp. To download it, click the button in the SuperCheckPartial pane in the right column of the Aggregator.

“Paranoid” validation is a two-edged sword. Because the latest master file is only compiled before major contests, it is virtually certain to omit new special calls and DXpeditions. You’ll have to decide for yourself whether you want to exclude them from the spots you send. Experience indicates that either Normal or Aggressive validation, set in your Skimmer or SkimSrv, provides quite adequate protection against busted spots.

At the bottom of the pane you’ll see confirmation that you’re connected to the RBN. Also included, and new in this version, the server can request only certain spots, and exclude bands of frequencies. As shown here, beacon spots will now be forwarded to the RBN, and spots in certain frequency bands will not. The idea is to avoid busted spots by excluding the main RTTY frequencies on the higher bands. This is a bit of an experiment, because in major contests the boundaries are often ignored. For that reason, any excluded frequency bands will likely be dropped from the server’s instructions during contests. You’ll note that in the full-screen image above there are no excluded frequencies shown

Once you are using the new Aggregator, please uncheck the "Post Only CQ Spots" option on the Telnet tab of your Skimmer or SkimSrv. That will send everything to the Aggregator, and it will decide which spots to forward. This will facilitate experimentation with new features, while not loading the RBN server with unwanted spots, and it’s the only way beacon spots can get to the RBN.

Again, the green text at the bottom of the pane tells you that all is well, and changes if there is something that needs to be corrected.

Moving on to the bottom left pane …



This is an added feature that some users will find useful. It lets you feed spots locally from the Aggregator to a logging program, and choose whether you want to see everything locally or only spots that have passed through the Aggregator's filters and been sent to the RBN. Again, the text at the bottom tells you the status of this feature - when a local "user" is connected, the text changes to green and reports that the connection has been made.

The top right pane of the Aggregator window is another new feature, to be used for messages from the server to Skimmer ops. It might be used, for example, to advise that a new Aggregator has been released, or a new blog article, or to provide information to help correct some issue that has arisen. It will be worth checking here periodically to see if there’s anything new.

The next pane tells you whether your master.scp file is the same one available from the RBN server, and provides a Download button that is greyed out unless a newer one is available.

And finally, the last pane.

It gives you all the information collected on each spot. Note, though, that it displays the spots as they are provided by Aggregator to the local Telnet port. In this case, I have SkimSrv sending all spots to the Aggregator, be they CQ, DE, or undetermined. The program will take care of sending only those spots that the server wants.

So that's the story. We hope you will all thank Dick and Felipe for their contributions, by downloading and using the new Aggregator.

73, Pete N4ZR

Men Working - Pardon Our Dust

You may have noticed in the last 10 days or so that the RBN has been down occasionally. Sometimes Skimmers can't connect, sometimes they connect but no spots flow, and sometimes the Telnet servers aren't working.

Alas, these are the inevitable side-effects of our efforts to make sure the RBN is reliable by the time the major CW contests come around. We're streamlining the database code and perfecting auto-restart functionality so that when the record loads we anticipate for CQWW CW and SS CW come online, we'll be able to handle them. The software guys are in two time-zones, and have to fit this around their real jobs, so we can't predict when these outages will occur.

If any software developers want to test their software against a simulated flow that is 50 percent above last fall's highest average, please contact me - n4zr@contesting.com and I'll turn on "the flood."

73, Pete N4ZR

New Aggregator - A Big Step Forward

Thanks to the programming skills of Dick, W3OA, we now have a new version of the Aggregator, the software that collects CW Skimmer or Skimmer Server spots and forwards them to the Reverse Beacon Network's server. The highly-stable beta (version 0.92b) is available for download here. Several of us have been using it for a month without a single problem, and it only takes about 5 minutes to get going. It offers several very significant new features, and we urge every RBN spot contributor to download and begin using it as soon as possible. If you encounter any bugs, please report them to w3oa@roadrunner.com; for operational questions, please write to me (n4zr@contesting.com)

There is one system requirement for the new Aggregator - you must have Microsoft .NET 4.0 installed on your computer. You can get it here, and it takes only 5 minutes to download (on broadband) and install. This is the reason why the Aggregator is only 90 KB - and since it is increasingly a requirement for other software as well, we hope this won't be an obstacle for anyone.

For users of Windows Vista and Windows 7, you will need to put the new Aggregator somewhere that the operating system will allow it to write an .ini file. That means installation almost anywhere but the Program Files directory - many of us use C:\Ham Radio as an alternate location, to avoid these issues.

Some security software (Norton Security's SONAR, for one) will try to block Aggregator from running, and even removing it from the installed location. These packages make their "decisions" based on what they regard as "risky software behavior." We can't really avoid getting them stirred up, given the way that Aggregator operates. If you run into this, you will need to tell the busybodies that you know best; all these security software packages have provision for reversing their "decisions."

Here is a screenshot of the new Aggregator - probably the best way to explain its features.





The biggest thing you'll notice is that the new Aggregator is a "real" Windows program, which means that you do all the setup on a single screen, rather than having to set command-line options. Despite that, it is very small,uses essentially no CPU power, makes no changes to the registry, and does not have to be installed.

The upper panel is the initial setup



These settings should normally work for your installation. 127.0.0.1 is the IP address of the computer the Aggregator is on, which must be the same one where you are running Skimmer or SkimSrv. The port number must correspond to the one you have set on the Telnet tab of Skimmer or SkimSrv. If you have set controls on access to your Skimmer or SkimSrv, by username or password, you'll need to set those to match. If you lose your connection to Skimmer for any reason, the green message at the bottom of the pane will change to red to alert you.

The next pane down controls what spots are sent to the Reverse Beacon Network. This is where things get really interesting.



The new Aggregator provides the functionality of the "Paranoid" validation setting in CW Skimmer, which users of SkimSrv may have been missing. If this option is selected, only spots that match calls in a master file are forwarded to the RBN. The Aggregator uses the text version of the widely-used master.dta file, called master.scp. You can download it here. Be sure you put it in the same folder (directory) as the Aggregator.

If you use the first option ("Send all spots RBN server requests"), but the Aggregator only forwards those spots requested by the RBN. Why should you care? Well, that means the Aggregator is responding to direction from the server - currently, that means it will send only CQ spots on HF, but all spots on VHF. In the future, we anticipate adding beacons and other non-CQ spots to the list that the server requests.

Once you are using the new Aggregator, please uncheck the "Post Only CQ Spots" option on the Telnet tab of your Skimmer or SkimSrv. That will send everything to the Aggregator, and it will decide which spots to forward. This will facilitate experimentation with new features, while not loading the RBN server with unwanted spots.

And again, the green text at the bottom of the pane tells you that all is well, and changes if there is something that needs to be corrected (your Internet connection is down, or the master.scp file is not found.

Moving on...



The bottom pane is an added feature that some users will find useful. It lets you feed spots locally from the Aggregator to a logging program, and choose whether you want to see everything locally or only spots that have passed through the Aggregator's filters and been sent to the RBN. Again, the text at the bottom tells you the status of this feature - when a local "user" is connected, the text changes to green and reports that the connection has been made.

The final pane (on the right) is basically a confidence-builder, to confirm that the Aggregator is getting spots from Skimmer or SkimSrv.



It gives you all the information collected on each spot. Note, though, that it displays the spots as they arrive at the Aggregator. In this case, I have SkimSrv sending all spots to the Aggregator, be they CQ, DE, or undetermined. The Aggregator will strip out all but the CQ spots (on HF).

So that's the story. We hope you will all thank Dick for his contribution, by downloading and using the new Aggregator.

73, Pete N4ZR

The Pluses and Perils of Updating

Recently, I contacted all of the Skimmer operators who had recently contributed to the RBN and were running anything other than the most recent versions of CW Skimmer or Skimmer Server. The reason for doing this was to encourage them to update because 1) it's free and 2) there was a bug in earlier versions that caused some of the "False CQ" spots that have plagued us.

On the theory that "no good deed goes unpunished," it turns out that updating to the most recent versions of CW Skimmer and Skimmer Server may undo all the good efforts people made to correct their receivers' calibration, if you are using Windows 7. The reason for this is that the .ini file in which the frequency calibration data is stored, which used to be located in the program directory, was moved in the most recent versions to an "application data" directory. What this means is that if you added a line for Frequency Calibration to your .ini file in the program directory, that correction may have been lost when you updated. If you are using Windows XP, nothing has changed, and you don't have to do anything.

For Windows 7, here's the solution, or rather two approaches to it. First, you need to identify where your new CWSkimmer.ini or SkimSrv.ini file is located. One way is to start CWSkimmer, open the Help menu, and select Data Folder. This opens Windows Explorer and the folder where the CW Skimmer.ini file is located. Click on the up arrow in the top of Windows Explorer, to go up one level in the path, and you will see folders for both CWSkimmer and SkimSrv. The .ini files are located in these two folders, which are sub-folders under Application Data..

The other way is to use Windows' Search feature to find the .ini file you are interested in. If you go this route, just search for either CWSkimmer.ini or SkimSrv.ini. Be sure to set your Search options so that the search will include system and hidden files. You will find two - one in the program folder and the other in an Application Data sub-folder - the one in the program folder is your old one, and the one in the Application Data sub-folder is the one that is currently active.

Typically, Windows 7 users have reported that the .ini file winds up in C:\Users\[username]\AppData\Roaming\Afreet\Products\CwSkimmer or C:\Users\[username]\AppData\Roaming\Afreet\Products\SkimSrv.

Now, the fix. You can either copy the relevant .ini file from the program folder to the Application Data sub-folder, replacing the one that the new installation put there. That will transfer all your settings, including your Frequency Calibration line. Or you can simply copy that line into the new .ini file. Restart Skimmer or SkimSrv and you should be back on frequency.

A New Way to Access the RBN, and Why You Should Care

A couple of months ago, we added an AR Cluster node to the DX Spider node that was the original source of Reverse Beacon Network spots. It hasn't attracted a lot of attention, until now. During WPX CW, some RBN users in Europe reported problems with Win-Test due to the high volume of spots, even when limiting on the DX Spider node to only a single country of origin.

Dave Zeph, W9ZRX, who signed his post "the PC guy at K3LR", responded to a cry for help on the Win-Test support reflector. His response is reproduced below, with his permission, in the hope that others will find it as useful as I did.

Here's Dave:

Hi Marek .... there are two different Reverse Beacon Ports - one supported by the DXSpider software, and another that uses AB5K’s V6 AR-Cluster software. This can be accessed via ArCluster.ReverseBeacon.Net Port 7000.

The V6 ArCluster Server has a very powerful SQL Filter and a command to ignore Duplicate Skimmer Spots - which is what I believe you want.

The documentation showing the Filter Commands can be downloaded from:

http://www.ab5k.net/ArcDocsVer6/UserManual/ArcDx.htm

Here are some of the Filter Commands K3LR sends to the ArCluster.ReverseBeacon.net Port 7000 Server. Note that the following Filter Strings limit the displayed Countries to just K and VE, and Filter DX Cluster Spots as well as skimmer Spots.

CW:

SET DX FILTER (Not Skimmer or (Skimmer and SkimCQ and not SkimDupe)) AND (Cty<>K AND CTY<>VE) AND (Freq >= 1800 AND Freq <= 1880 OR Freq >= 3500 AND Freq <= 3600 OR Freq >= 7000 AND Freq <= 7100 OR Freq >= 14000 AND Freq <= 14140 OR Freq >= 21000 AND Freq <= 21130 OR Freq >= 28000 AND Freq <= 28130)

PHONE: (No Skimmer Spots Wanted)

SET DX FILTER ((Cty <> K and CTY <> VE) AND (Freq >=1800 AND Freq <= 1900 OR Freq >= 3600 AND Freq <= 3900 OR Freq >= 7000 AND Freq <= 7300 OR Freq >= 14150 AND Freq <= 14350 OR Freq >= 21200 AND Freq <= 21450 OR Freq >= 28300 AND Freq <= 29100)) OR CALL = K3LR

To Just See Specific Skimmers: (In this example either K1TTT-# or W3LPL-#)

SET DX FILTER (Not Skimmer or (Skimmer and SkimCQ AND (SPOTTER = K1TTT-# OR SPOTTER = W3LPL-#))) AND (Cty<>K AND CTY<>VE) AND (Freq >= 1800 AND Freq <= 1880 OR Freq >= 3500 AND Freq <= 3600 OR Freq >= 7000 AND Freq <= 7100 OR Freq >= 14000 AND Freq <= 14140 OR Freq >= 21000 AND Freq <= 21130 OR Freq >= 28000 AND Freq <= 28130)

Give these a try. The SkimDupe Filter was one added by Terry after the CQ WW CW, and subsequently used in the ARRL CW Contest – for precisely the reason you mentioned. Skimmer Spots for a station were being overwritten in the ANNOUNCMENTS and BANDMAP Windows by Duplicate Spots from additional Skimmers. This caused a lot of “movement” or “churning” in those windows and was a real distraction.

So that's the story. Thanks to Dave and Terry, users have a new tool, and another node for the RBN.

73, Pete N4ZR

Auto Band Changing with SkimSrv and the QS1R

One of the limitations of using SkimSrv (Skimmer Server) with the QS1R receiver is that you can only monitor 7 bands at a time. Yes, only - because if you want to cover 80 and 160 at night, and 12 and 10 during the day, or listen for sporadic E on 6, then you have to remember to change the band selections in Skimmer Server.

Because I keep forgetting to do this manually, I wanted a way to make changes automatically. Following is my simple-minded solution.

1. Download AutoHotkey and install it on your computer.
2. Use Notepad to view and store these two scripts. Be sure to give the file-names the extension .ahk so that Windows will know to run AutoHotkey when you double-click on the scripts.

To start 12 and 10, disabling 160 and 80:

;This script will tell SkimSrv to stop listening on 80 and 160, and to start listening
; on 12 and 10
winActivate Skimmer ;tells Windows to apply the following in the SkimSrv window
Click 73, 35 ; select the Skimmer tab
Click 34, 178 ; click on 160 to deselect it
Click 34, 162 ; click on 80 to deselect it
Click 37,297 ; click on 12 to select it
Click 33, 277 ; click on 10 to select it
Click 290,438 ; click on Apply
Click 27, 42 ; Select the Status tab


Similarly, to start 80 and 160, disabling 12 and 10

;This script will tell SkimSrv to stop listening on 12 and 10 and to start
; listening on 80 and 160
WinActivate Skimmer ; makes sure that the following apply to the SkimSrv window
Click 82,36 ; Select the Skimmer tab
Click 37,297 ; shut down 10
Click 33, 277 ; shut down 12
Click 34, 178 ; start 80
Click 34, 162 ; start 160
Click 290, 438 ; Click Apply
Click 28,32 ; select the Status tab

It's probably obvious, but what we are doing here is telling the cursor to go to a location in the SkimSrv window and then click there. Because the locations are defined in relation to the window, and because the window cannot be re-sized, this works wherever the window is on your screen, but it must be open when the script is activated.

I figured out what coordinates to show in the Script by using the AutoIt Windows Spy which comes with AutoHotkey

Once you have the two scripts stored, double-click each one to make sure it does what you intend. Once things are working, you can customize the scripts for your own particular needs. Incidentally, it is critically important that you leave a space before the ";" that denotes a comment, when the comment is in-line with a script command. Otherwise, AutoHotkey will try to run what appears after the semi-colon.

Now, go to All Programs>Accessories>System Tools>Scheduled Tasks, and set up your scripts to run every day when you want them to. The only trick here is that the account from which you run the Task Scheduler must have a password - blank will not work. Set the same password for each script.

Now I have no excuse for forgetting to switch band coverage!

73, Pete N4ZR

Frequency Calibration, the Easy Way

Frank, W3LPL has introduced me to a really easy way to calibrate your RBN CW Skimmer or SkimServ. It goes like this.

1. Set a filter on http://reversebeacon.net to pass only spots from your continent, and only on 12 and 10 meters. The reason to use these bands is that any error in calibration increases as you go up in frequency.

2. Wait till the time of day when those bands open, and compare the frequency reported by your Skimmer with those reported by others in your area. The vast majority of Skimmers in Europe and North America are accurately calibrated now, so that if your calibration is off it will probably stand out among multiple spots of the same station.

3. Calculate a correction factor by dividing your displayed frequency by the true frequency. For example, if you spot a station at 28005.4, and the majority report 28005.1, then the resulting correction factor would be 1.00001071.

4. Now open either CWSkimmer.ini or Skimserv.ini, depending on which Skimmer software you are using. You can use Notepad for this purpose.

5. See if you have a line that reads "FreqCalibration=....". If you do not, then add that line, with your correction factor from step 3 inserted to the right of the "=" sign. If you do, then multiply the correction factor by the number you find there, and enter the new number instead.

6. Now save your change, close Notepad and close your Skimmer. When you reopen the correction will be added.

7. Recheck your calibration. You should be very close. If you are still off in either direction, repeat the steps in 3-5 above, making sure to multiply this new correction factor by the value already in the .ini file to derive the new FreqCalibration value.

That's all there is to it. Please check your frequency calibration and help us build the RBN's reputation for producing accurate spots for all users.

Thanks and 73,

Pete N4ZR