NEXRAD Rainfall Process

Products Derived from National Weather Service NERFC

Hydrometeorological Discussion

Snow Page

NEXRAD Data

The stage I, stage III and MPE data files are collected from the RFCs. The files collected contain a day's worth of data which is made up of UNIX compressed hourly files that are tarred into a single file. The daily files are then tarred together to create a monthly file which is available on the web. The table below shows the date range of the archived data.
Quick descriptions of the data files for stage I data and stage III data are available. For more detail information concerning data formats and decoding procedures, see Precipitation Processing System, NEXRAD Stage III Precipitation Data and HRL Distributed Modeling Research online documentation. (Note: Please view the National Weather Service Disclaimer regarding the information on this US Government Server.)
WSR-88D Files

DS.81dpr/ is the product of 1 hourly digital rainfall. The U.S. National Weather Service provides anonymous FTP access to files containing collectives of code generated and binary radar imagery products from NWS, FAA, and DOD WSR-88D sites located in the United States and Puerto Rico. The collectives are in the same format as received from the WSR-88D Radar Product Generator (RPG). The server internal structure for sub-directory names and file names can be found with descriptions in the document TOC File Name Standards. The naming conventions for the sub-directories and files are described in the document titled: File and Directory Name Standards. The data is available at the NWS Telecommunication Gateway on the server ftp://tgftp.nws.noaa.gov via anonymous FTP.
Decoders are available for WSR-88D products
Data available from HDSS Access System

Quantitative Precipitation Products

RFC Data archive

GIF imagery of all storm events to determine whether retrieval of Stage III data.

NEXRAD Viewer Process

View Historic Storms
Retrieve Data Map
Perform analysis and exports

TRMM data for online visualization and analysis (TOVAS)

Stage I Data
Stage I data comes in two types: encoded and decoded. The encoded products are the hourly digital precipitation (HDP) products which are transmitted to the RFC via AFOS. After the receipt of the encoded products, the RFC decodes and stores the information. The decoded products contain only a subset of the encoded products and so NHDS stopped archiving the decoded products as of February 1996. Only the encoded products are available on the web.

Stage III Data

San Juan, PR

o Multisensor Precipitation Estimator Data

Displaying and Using NWS XMRG/HRAP Files
within ArcView or Arc/Info GIS

Created: 11/26/1999
Last Modified: 1/28/2002

BACKGROUND

ESRI Software Built-in Projection Capabilities (Differences between Arc/Info and ArcView)

Approaches for displaying XMRG/HRAP data in ArcView

Scenario 1. I have Arc/Info software as well as ArcView 3.1 with the Spatial Analyst. I want to display XMRG/HRAP precip files as Grids in ArcView and overlay vector data. I have access to Arc/Info software as well as ArcView 3.1 with the Spatial Analyst.
- C Programs for XMRG to Grid conversion
- An Arc/Info Projection file for HRAP

Scenario 2. I want to display XMRG/HRAP precip files as Grids in ArcView and overlay vector data. I only have ArcView 3.1 software with the Spatial Analyst extension (No Arc/Info).
- An ArcView Extension for working with HRAP Coordinates

Scenario 3. I want to determine which HRAP cells fall within my basin or display HRAP-based data in another coordinate system (I do not have Arc/Info).

BACKGROUND

Several people have asked: How can I display and use XMRG files in ArcView? This web page is intended to answer questions about XMRG, HRAP, and ArcView. XMRG is a binary file format used within the National Weather Service to store gridded data. More specifically, gridded rainfall products like NEXRAD StageIII are sometimes stored in the XMRG format. HRAP (Hydrologic Rainfall Analysis Project) is a grid coordinate system used within the National Weather Service. One way that the HRAP grid is used is to define the location of StageIII rainfall cells. IMPORTANT POINT: XMRG is a file format and HRAP is a coordinate system. Gridded data that is not referenced to the HRAP coordinate system could be stored in XMRG format. Gridded data referenced to the HRAP coordinate system may be stored in other file formats (e.g. netCDF and GIF). This page describes several programs that make it easier to work with data referenced to the HRAP grid and stored in XMRG format. Several of these programs are also useful if you have data referenced to HRAP in a non-XMRG format.

The paper: Reed, S.M., and D.R. Maidment, "Coordinate Transformations for Using NEXRAD Data in GIS-based Hydrologic Modeling," Journal of Hydrologic Engineering, 4, 2, 174-182, April 1999, offers details about how the HRAP coordinate is defined and insights as to why displaying HRAP grids with GIS data has been a point of confusion. It is not necessary to read and fully understand this paper to correctly apply the programs provided this page (although it is helpful to have a basic knowledge of map projections and how they are implemented using ArcView and Arc/Info). For those of you who have looked at this paper, there are a few misleading statements that could not be corrected before publication. The discussion revolving around Figure 6 in this paper incorrectly implies that radar data are mapped by drawing a circle representing the radar range in the HRAP plane. In the actual NWS radar software, the longitude, latitude coordinates corresponding to radar estimates are computed on a radar-centric polar grid using equations for a spherical earth prior to projecting data into the HRAP coordinate system. The HRAP grid is used primarily for mosaicing and displaying rainfall values. As long as distances are not measured in the HRAP plane, the "scale factor" problem described in relation to Figure 6 is not an issue. With this out of the way, the main point of this page is to provide tools to work with data sets referenced to the HRAP grid using the standard GIS software packages ArcView and Arc/Info.

ESRI Software Built-in Projection Capabilities (Differences between Arc/Info and ArcView)

The methods and programs described on this page were devised keeping in mind the capabilities inherent to ESRI software (Arc/Info and ArcView), taking advantage of these capabilities where possible. Arc/Info 7.0 or higher provides excellent support for map projections and coordinate transformations. ArcView 3.1 is more limited in that it only allows projection of vector data (not raster data i.e., Arc/Info Grids) and has more limited built-in flexibility in defining input and output projection parameters. Even projecting vector data Themes in ArcView 3.1 is not part of the base software functionality (Note that when I say projecting data Themes I mean actually creating a new data set, not redefining how Themes are displayed in a View); however, some Theme projections can be done using the free "Projector!" extension distributed by ESRI or using Avenue. ArcView 3.2 is supposed to have improved support for map projections, but it is unclear whether the ability to project Grids will be included in future versions of the Spatial Analyst.

Approaches for displaying XMRG/HRAP data in ArcView

The common problem that is being addressed here is that that precipitation grids are often available in XMRG format and referenced to the HRAP grid, and it is often desirable to display and/or analyze these precipitation grids along with other data sets (e.g. state boundaries, watersheds, streams, gage locations) that are not commonly available in the HRAP coordinate system. To provide a consistent framework for analysis, either the rainfall grids or the "other" data must be re-projected. It may or may not be desirable to keep the precipitation data in a gridded file format.

Here are some scenarios that the programs described below will support. Certainly, other scenarios may be more appropriate depending on the intended application and available software.

Scenario 1. I have Arc/Info software as well as ArcView 3.1 with the Spatial Analyst. I want to display XMRG/HRAP files as Grids in ArcView and overlay vector data. I have access to Arc/Info software as well as ArcView 3.1 with the Spatial Analyst.

Scenario 2. I want to display XMRG/HRAP precip files as Grids in ArcView and overlay vector data. I only have ArcView 3.1 software with the Spatial Analyst extension (No Arc/Info).

Scenario 3. I want to determine which HRAP cells fall within my basin.

Scenario 1. I have Arc/Info software as well as ArcView 3.1 with the Spatial Analyst. I want to display XMRG/HRAP precip files as Grids in ArcView and overlay vector data. I have access to Arc/Info software as well as ArcView 3.1 with the Spatial Analyst.

The first step in both Scenarios 1 and 2 is to convert an XMRG file (in the HRAP coordinate system) to Arc/Info Grid format. A C program called xmrgtoasc.c can be used to translate XMRG files to an ASCII file format that can be imported into ArcView. xmrgtoasc.c creates a file with the ".asc" ending. The program takes three arguments: input file name, output file name, and the key word "ster" or "hrap."

Example syntax used to compile on HP Workstations: cc -Aa -o xmrgtoasc xmrgtoasc.c
Example execution syntax: xmrgtoasc <infilename> <outfilename> ster

The resulting files can be loaded into ArcView by clicking the File--> Import Data Source menu item when the Spatial Analyst is loaded and a View is active. Select "ASCII Raster" as the import file type.

The HRAP grid is defined in the plane of a polar stereographic map projection with the following parameters:
longitude of the projection center = -105, standard (true) latitude = 60 N. A spherical earth of radius 6371.2 km is assumed in defining the HRAP coordinate system. Using "ster" as the third argument to the xmrgtoasc.c program generates header information in Polar Stereographic coordinates with units of meters. The "hrap" argument generates header information in HRAP units. These Polar Stereographic coordinates are related to HRAP coordinates as follows:

xster=hrapx*4762.5 - 401*4762.5
yster=hrapy*4762.5-1601*4762.5

Having grids available in Polar Stereographic coordinates makes it possible to use the Arc/Info Project function to project gridded data; however, this requires tricking Arc/Info. (Thanks to Tom Evans at HEC for suggesting this approach.) The HRAP projection is defined using a sphere of radius 6371200 m and using a true latitude of 60 N. Arc/Info does not support customizing both the sphere radius and the true latitude when using the Polar Stereographic projection. The default radius for a sphere used by Arc/Info is 6370997 m. The "trick" is to use the Arc/Info default radius and a true latitude that is slightly different than 60 N as shown in the projection file below to approximate the HRAP definition. This projection file closely approximates the conversion from the HRAP defined Polar Stereographic coordinates to geographic coordinates. This is particularly useful when dealing with grids because it allows a user to take advantage of the resampling capabilites of Arc/Info when projecting grids. Differences between using this "trick" projection file and the exact equations are negligible at 33 N (~0.4 m differences were found when comparing these results to the exact transformation for single points). It appears that this trick cannot be used within Avenue to project Shapefiles because ArcView doesn't support a polar Stereographic projection where you can specify the true latitude. Programs for transforming Vector data using Avenue are described below in Scenario 2.

/* Example projection file
input
projection polar
spheroid sphere
units meters
parameters
-105 0 0                        /* longitude of the center of the projection
60 0 24.5304792            /* true latitude dd mm ss
0.0                                /*false easting
0.0                                /*false northing
output
projection geographic
spheroid sphere
units dd
parameters
end

Scenario 2. I want to display XMRG/HRAP precip files as Grids in ArcView and overlay vector data. I only have ArcView 3.1 software with the Spatial Analyst extension (No Arc/Info).

Under certain circumstances, it may be desirable to leave the gridded data in HRAP or Polar Stereographic coordinates and convert reference vector data sets to this coordinate system. To implement this scenario:

- Use one of the C programs described in Scenario 1 (xmrgtobin.c or xmrtogasc.c) to create Arc/Info Grids from XMRG files in HRAP coordinates

Note: ArcView supports a Polar Stereographic projection but not the type used by HRAP which requires specification of a "true" latitude.

- Project Point, Line, or Polygon shapefiles from geographic coordinates to HRAP coordinates using scripts provided in the coord.avx extenstion. (Place this file in the ArcView/ext directory (PC or UNIX) or your home directory(UNIX)). If you load the extension coord.avx (Listed as "Sp-coord" in the Load Extensions dialog) you will see one new Menu called "HRAP" with one Item -- "Create HRAP Center Points," two new Buttons "G/H" and "G/A," and a new Tool Menu with the Tools "H/G" and "G/H." The G/H button will project active Point, Line or Polygon Shapefile Themes into the HRAP coordinate system. Note, this program may take a while for large data sets because it cannot take advantage of pre-compiled Avenue requests.

Note: The H/G and G/H Tools can be used to click a point on the map and return the geographic coordinates of the point clicked if the display is in HRAP ( H/G ) and return the HRAP coordinates of the point clicked if the display is in geogaphic ( G/H ). Other features of this Extension are described in Scenario 3.

Scenario 3. I want to determine which HRAP cells fall within my basin or display HRAP-based data in another coordinate system (I do not have Arc/Info).

Scenario 3 stems from my work on Threshold Runoff (threshR) where it is desirable to do the basin delineation and spatial analysis in an Albers Equal-Area projection. For threshR, it is necessary to know the location of the center points of HRAP cells in the Albers Equal-Area projection. To do this, a Shapefile of HRAP center points is created and projected from HRAP to geographic (lon-lat) and from geographic to Albers.

1. Using the coord.avx extension described in Scenario 2, select the Menu item HRAP --> Create HRAP Center
Points. This program gives the option to create a Shapefile of HRAP center points in HRAP coordinates, polar Stereographic coordinates, or geographic coordinates (lon-lat). The extent of the Shapefile to be created can be specified by (1) longitude and latitude extent or (2) the lower left HRAP coordinates and the number of columns and rows (The second approach is preferred because of issues explained in step 3 below).

2. (Skip this step if you want to work in geographic coordinates.) Create a (lon-lat) center point shapefile in Step 1 and project this into an Albers Equal-Area projection using the Button G/A. Note that specific Albers parameters are hard coded into this program. For ThreshR, the locations of the HRAP center points are needed so that threshold runoff values can be interpolated to these points. The center point coverage can also be used to automatically identify the HRAP cells falling within a basin. It is also possible to join XMRG data values to the center-point Shapefile to graphically display storms as described in Step 3.

3. To join XMRG data values to an HRAP shapefile, a comma delimited text file with at least two columns must be created. One column contains a unique ID to join to the HRAP shapefile and the other a set of XMRG data values. A C-program called xmrgtolist.c will create a comma delimited text file from an XMRG file. The compile syntax is in the file header and the run syntax is: xmrgtolist <infile> <outfile>. (Note: you should give the "outfile" a ".txt" extension). The xmrgtolist program creates unique IDs for all hrap cells by numbering from the lower left corner, across columns, up one row, etc. The ArcView menu item Create HRAP Center Points (Step 1) uses the same numbering scheme; however, to be able to join the values in the ".txt" file to the center-points Shapefile, the center-points shapefile must be created with the exact same extent (number of columns and rows) as defined in the XMRG file, otherwise the join ID's may be inconsistent. Load the output from xmrgtolist into ArcView by clicking on Tables and then Add in the Project window. Join the resulting Table to the center-point Shapefile using Table-->Join. The XMRG data can now be displayed by manipulating the legend of the center-point Shapefile. The Shapefile could be converted to a grid at this point (faster display), but you can get a pretty good visually display using the point data alone.

ESF Work on Scan