New Features in Aggregator 2.3, now on the web site

W3OA has released Aggregator 2.3, after a series of beta tests.  It has a number of new features that will be of interest, particularly as we begin also providing digital spots from stations using DL4RCK's RCKskimmer.  The new release is now on the web site, under Downloads.

The big change in the new Aggregator is the addition of a new Combine Skimmers tab, illustrated below:





As usual with Dick's programming, the new tab is self-explanatory.  It will take a stream of combined Skimmer spots from multiple Skimmers, coming from wintelnetX or AR Cluster Serve,r and feed them to the "mother ship" under a single callsign, substituting what you fill in here for the information normally provided separately by each instance of CW Skimmer or Skimmer Server.

The other big change is that Aggregator 2.3 now supports RCKskimmer by DL4RCK for digital modes by accepting mode and speed information from it and forwarding the information to the server.  From there spots, in a new format are distributed via our two Telnet servers.  They look like this:

DX de KM3T-#:    21016.5  SM7YIN         CW    09 dB  25 WPM  CQ      1233Z
DX de EA4TX-#:   21016.6  SM7YIN         CW    22 dB  25 WPM  CQ      1233Z
DX de KQ8M-2-#:  21071.4  RA3PS          BPSK  08 dB  31 BPS  CQ      1233Z
DX de LA5EKA-#:  14028.1  OZ3NP          CW    11 dB  22 WPM  CQ      1233Z

The format is compatible with all standard DX clusters, since all of the data - from mode through CQing status - is contained in the Comment field size set by the basic spot format.

It is our intention to support any further digital Skimmers that emerge, so long as they deliver their spots by Telnet and their output conforms with that provided by CW Skimmer, Skimmer Server and RCKSkimmer.

We urge all Skimmer operators to update to Aggregator 2.3 as soon as possible.

73, The RBN Development Team

Aggregator 2.2 Released

Dick, W3OA has released the latest Aggregator after a substantial period of beta testing.  Let's have a look at the key new features, and then I'll show you some screenshots.

• The Status tab now includes information for the individual Aggregator op about how accurate the frequency of his spots is currently, as measured against both fixed and dynamic standards on each band.  Note, this is not foolproof.  For example if a CQing station QSYs a short distance and calls CQ again, the Aggregator may temporarily report  a big skew (our term for a frequency discrepancy).  Skew figures are recalculated and sent to each user every 5 minutes, so if you see one that is surprising, just wait 5 minutes and see if it disappears.  With the QS1R receiver, a calibration problem will typically involve excessive skews only on the higher bands, and they will be progressively larger as the frequency increases.  If you think you have a calibration problem, check out this blog post from last year for a quick and easy way to calculate the correction factor to put in your .ini file.
• Operators can now select the port number they want for local Telnet connections.
• The Aggregator now shows the validation level set in CW Skimmer or Skimmer Server, and recommends using Normal if not set to Normal.  This is not mandatory, but does help to maximize the number of spots forwarded with seemingly negligible effect on accuracy.
• Popup windows warn the user if the Aggregator can't connect either to the Skimmer or to the RBN Server for 5 minutes.  Another popup will suggest updating if you are not using the latest Aggregator available on the RBN web site.
• VHF+ spots than look like grid squares can now optionally be filtered out rather than being forwarded to the server.
• VHF+ spots with SNRs <+ 1 dB can now be filtered out, to prevent bogus spots of various sorts of spurious signals.

The big changes are summarized on the Status tab:

The red box is just for emphasis, and doesn't actually appear in the Aggregator.  It points out where the frequency calibration summary will be found.

Next up is the Spot Filters tab:

This one is all pretty self-explanatory.  Dick's done a terrific job of making each option pretty self-documenting.  This is where you set up Bad Call lists, useful if RFI in your station causes a lot of false  spots that are corrupted versions of your callsign.  You can also set up a list of Notched Frequencies, and of course the check boxes at the bottom of the page let you set up those lists and then decide whether or not to use them at any particular moment.

And finally, the last tab with changes is the Connections tab:

Again, the red is just for emphasis in this blog.  The rectangle shows much more detail on your frequency calibration, updated every 5 minutes.  The ellipse shows where you can now select the local user port.

So that's the story.  What do you think?

73, Pete N4ZR

About This Project - redux

Recently, I started thinking that I'd better capture some of the history of the RBN before passage of time and an aging memory obscured it forever.  The About This Project section on the RBN website needed updating anyway, so I collected a few key dates from e-mails and Skype logs, and here it is.  I hope you enjoy it.

About This Project

The Reverse Beacon Network was borne out of an e-mail exchange in March of 2008 between PY1NB and N4ZR.  Felipe had been running a unique DXing web site, DXWatch.com, for several years, while Pete had been working with VE3NEA, the author of the CW Skimmer software, since late 2007 to test, develop and refine it.  Felipe saw a way in which the basic framework of DXWatch could be adapted to display Skimmer spots at a central location as “reverse beacons”, spotting everything they heard.  Early in April, Felipe wrote the first Aggregator software, intended to receive spots from Skimmer’s Telnet server and transmit them to the web site for display.

The web site was initially the only way to view Skimmer spots. But as the controversy raged in the contesting community over whether unassisted single operators should be allowed to use this new technology, it occurred to the RBN operators that there might be an opportunity here to contribute Skimmer spots to the worldwide contest and DX community through a Telnet server using DX cluster software.  We had no clue how quickly this would change contesting.

It took a while, but in April 2010 the Telnet server debuted.  Almost immediately, it proved very popular, to the point where the server began to buckle under the load of being the only outlet for RBN spots  

At first, there was general consternation at the thought of Skimmer spots being integrated into the traditional DX cluster structure, for fear that their sheer volume would submerge the traditional spots people were used to.  Happily for the RBN, though, writers of cluster software soon recognized that they could accommodate the RBN spots by providing filters to segregate the Skimmer spots if desired.

Meanwhile, Nick, F5VIH/SV3SJ joined the RBN team.  His computer science background was a great asset, and in July 2010, he rolled out the Signal Analysis Tool, a graphical way to compare signals of multiple stations on multiple bands, as heard by a single Skimmer anywhere in the world. 

 

In September 2010, VE7CC and VE1DX began distributing RBN spots through their cluster servers.  Shortly thereafter, AR Cluster Version 6 was released in beta with similar provisions and an advanced filtering scheme. 

 In November, just in time for CQWWCW, Dave, KM3T joined the team.  He, Nick and Felipe worked hard to ensure that the RBN servers would not fail during the contest.  They succeeded, and the Telnet server delivered over 1.7 million spots without incident.  In March of 2011, a second Telnet server running ARC6 was added to the RBN’s facilities, spreading the load and allowing for distribution of Skimmer spots to ARC6 clusters worldwide.

In September 2011, Dick, W3OA joined the team and wrote the first Windows Aggregator.  The beta was a success, and in succeeding months he delivered increasingly sophisticated versions of the software, which is now in release 2.1.  In November 2011, the RBN handled 3.25 million spots during CQWW, an average of 18.9 spots per second, with no problems.

So far in 2012, the RBN’s servers have been handling the load nicely. The  ARRL DX contest, the Russian DX Contest, and WPXCW passed without incident.  In WPX, the RBN actually handled slightly more spots than in last year’s CQWW, which gives us a sense of what to expect next year in CQWW.

Perhaps more importantly for the loading of the system, we topped 100 simultaneous Skimmers during the weekend, and actually had 114 unique Skimmers contribute  during that period.  As far as hardware is concerned, we’re in a period of watchful waiting.  At some point the database server will max out, probably necessitating separating it from the web server, but we seem to have a little way to go yet.  Meanwhile, Nick and Felipe are working on a new set of statistical tools that should enable everyone to get the numbers he needs from the system in near real time.  We’re optimistic that usership will continue to grow.

Stay tuned … who knows when the next good idea will come along?  And if you’d like to join us, drop us an e-mail and tell us what you have in mind.

73, Pete Smith, N4ZR

Some Thoughts on Accuracy

In order to correctly evaluate the accuracy of the RBN versus traditional spotting, I think you need to start with the idea that the RBN is not a traditional spotting network.  Here's what I mean.

Accuracy of a properly set up CW Skimmer in copying callsigns runs right around 99 percent.  There is an as-yet-unresolved problem with copying callers as if they were runners - more on that below.  But let's assume the 99 percent is right for an individual Skimmer, one not assailed by local RFI.

If you consider the RBN, on an open band, with no spot filtering by spotter location, then the picture changes.  Suppose you have 30 Skimmers in zones 4 and 5, all copying an open band.  Each running station will be spotted every 10 minutes by each RBN station, so long as the station remains on its run frequency.  You could, theoretically, have as many as 180 spots of each running station every hour.  In that case, simple probability says there will be roughly 1.8 busted spots per hour of each one.

If you are sitting at a big multi-op, receiving spots from all over the world, or just all over the US, then it is almost inevitable that you will see busted spots a lot more often than from the Cluster network.  Not only are there 10-15 times as many spots, but there will be busts of many running stations, just by the math.  On Sunday, you will see tons of busts, because by that time you will have worked most of the good callsigns, leaving only the busts on your bandmaps.

There are several partial solutions, not all of which will be implementable at a big multi, but let me mention a few:

*  Filter by spotter location, so that you only get spots from stations who are probably hearing the same things as you can.  For example, I filter by spotterstate = MD or PA or VA or NC or WV (have to get my own spots too).  That cuts down the number of Skimmers feeding my bandmap to 7-8, sharply reducing the probability of busts.

*  Use the "Unique > x" filter in AR Cluster version 6.  That filter only passes a spot if at least x+1 Skimmers worldwide have copied the same spot the same way in a relatively short period of time.  This helps a lot to winnow out busts, as you can imagine.  The RBN has a very robust ARC6 node at arcluster.reversebeacon.net, port 7000.

*  Use a logging program that permits displaying unworkable spots (already worked, or not workable in a given contest) on the bandmap.  For example, N1MM Logger displays such spots in gray.  The advantage of doing this is that if LZ9W is running, but you worked him 30 hours ago, and LZ9WL suddenly shows up on the same run frequency, you can see that it's pretty likely to be a bust.  This also helps with callers mistakenly identified as runners; if you see calls appearing one after another on the same frequency as one of the big runners, you're probably safe in skipping past them.

We hope this is helpful.

73, The RBN Team

Aggregator 2.1 - new insight for Skimmer ops

The newest Aggregator, Version 2.1, is now available, after extensive beta testing.  This post explains the new features of this release, tab by tab.

First of all, there is an entirely new tab titled "Skimmer Traffic."  Here's what it looks like:

  Overwhelming, right?  What this does is to keep track of what happens to every spot made by your Skimmer.



The left-hand side of each row is exactly the same as you are used to seeing in the Aggregator, except for the color coding.  Only the green spots are actually sent along to the RBN.

The right-hand side is where the fun stuff is.

For example, take a look at the first green timestamp. The rest of that line (the brown part) tells you how Aggregator decided to send the spot on to the RBN, and what it sent. The first brown entry is the frequency sent to the RBN.  This will be different from the spot frequency on the left if you are using a transverter in front of your SDR and have entered a base frequency to be added to each spot.

Then Skimmer decided that first green spot was a "CQ" spot.  If it had been a VHF spot, Aggregator would have sent it on anyway, but NVHF means it was not. NExcl means that the spot's frequency was not within the excluded frequencies controlled by the server (more on that below), and NBcn means it was not a regular, listed beacon.  Either VHF or Bcn would have over-ridden an NCQ determination, while Excl would have blocked a spot that otherwise seemd to qualify (see below). NInMaster didn't matter, because I had not selected the option to spot only those stations in the master.,scp file, and finally, NInBadCall meant that I had not identified this call as a "badcall", one of those produced by local RFI.

Why bother?  Some RBN Skimmer Ops wanted to be able to see why a given spot was or was not sent to the RBN, or whether their BadCall list was working properly.  This should give them all the information they need.

Reminders about the color-coding and symbols are at the top of the Skimmer Traffic tab.

Some users had expressed a need for a way to notch out specific frequencies, typically on 60 meters, because digital signals on those frequencies were being mis-decoded.  This capability has now been added to the  Spot Filters tab.

On the .ini files tab, Edit buttons have been added for each of the files listed in the two .ini rotations.

There is a new feature in the middle panel of the Connections tab. 



The last check-box allows you to accept or reject a list of excluded frequencies downloaded from the RBN.  Typically, the purpose is to block "spots" of RTTY stations, where Skimmer will attempt to decode Baudot as if it were Morse.  However, during contests you would want to un-check this option, because CW contest activity typically runs into the normal RTTY frequencies and beyond.


In addition, in the Local User area at the bottom, Dick has added two new features on the right side. Port 7550 is now capable of accepting more than one logon at a time, in case several of your friends want to connect locally. 



 The SETT response is what Aggregator uses to tell the RBN server periodically what bands you are listening on.  We thought any local users might want the same information periodically.


That's all for this release.  What would you like to see in the next one?

73, Pete N4ZR

What Does a Flare Look Like?

Amazing screenshot from the RBN taken during today's flare.  Questions, anyone?