Extraterrestrial radiation dataset

I am currently helping supervise a masters student for their summer project. They’re working on improving a parameter used in flood estimation. The drive for the project is to use the revised parameter in the assessment of natural flood management, something SEPA and local authorities are now bound to consider since the implementation of the Flood Risk Management (Scotland) Act 2009.

The parameter derivation is dependant on a number of datasets and uses these to undertake a spatial regression, thus offering a new definition.

One dataset is a grid of potential evapotranspiration. For national assessments in the UK this is traditionally taken from the MORECS data owned by the Met Office. Unfortunately the cost of this was prohibitive (£4000 plus VAT for the UK) and as a core product, the Met Office were unwilling to release it with a student licence.

With a bit of hunting I came across this article by Kay and Davies, which derives a PE grid from freely available data and makes comparisons to MORECS. This seemed a viable solution and so became our chosen path.

To calculate the PE data extraterrestrial radiation is required. Kay and Davies’ paper provides a reference to Allen et al. who calculates this based on Julian day and latitude (calcs on pdf page 24, journal page 70). As this has potential use for my work (input solar energy probably has some impact on snow accumulation and melt!) and the student was struggling: I decided to roll up my sleeves and do some sums. You can see page 70 from the paper below, showing the required calculations.

Extraterrestrial radiation calculations from Allen et al. 1994

The work was completed using QGIS and R. These are freely available. Below are the steps I undertook and also the final datasets available for download.

• Created a 5 km grid to match the Met Office UKCIP09 one, because we’ll be using the 5 km temperature grid to calculate potential evapotranspiration (using QGIS Vector/Research Tools/Vector grid)
• Extracted the polygon centroids from the vector grid (QGIS Vector/Geometry Tools/Polygon centroids)
• Previous two steps were working in British National Grid, EPSG:27700. I converted the layer to WGS84 (EPSG:4326), as I needed latitude, not northing (QGIS save as…)
• Opened the new WGS84 file in QGIS and appended geometry columns (QGIS Vector/Geometry Tools/Export/Add geometry columns)
• Exported the file as a csv (QGIS save as…)

I next spent a bit of time writing a script in R to do the work. Initially I toyed with the idea of using a spreadsheet, but the thought of many fill downs and all the potential errors were not appealing. Most of the script writing time was spent working out how to get the matrices to appear in the sums correctly.

If you’re reading this and you think I’ve made a mistake – please tell me!

# import latitude
lat_raw = as.matrix(read.csv("~/file path/ext_rad_wgs84_cent.csv",sep=",",header=1))

# prepare latitute
lat_len = nrow(lat_raw)
lat_rad = lat_raw[ ,2]/180*pi
lat = matrix(lat_rad, nrow=lat_len, ncol=365)

# matrix of days, row number from lat
j = matrix(c(1:365), nrow=lat_len, ncol=365, byrow=1)

# calculate extraterrestial radiation
sigma = .409*sin(.0172*j-1.39)
ws = acos(-tan(lat)*tan(sigma))
dr = 1+.033*cos(.0172*j)
Ra = 37.6*dr*(ws*sin(lat)*sin(sigma)+cos(lat)*cos(sigma)*sin(ws))

# sum every day to get annual
Ra_ann = rowSums(Ra)

# add x,y columns
Ra_ann_output = cbind(lat_raw[,1:2], Ra_ann)

# output results
write.csv(Ra_ann_output,"~/file path/rad_ann",row.names=0)

The script outputs to a csv file with two columns of coordinates (lat-long in degrees) with a third column of annual extraterrestrial radiation. If you’d like the data, please get in touch and I’ll send it on. Unfortunately wordpress.com does not allow csv files to be hosted.

Ordnance Survey Terrain 50 elevation model

This is quite a technical post about (free) elevation data of Great Britain, analysed for Scotland.

I apologise for the screen shots, I wasn’t in the mood to make beautiful maps!

In spring 2010 the Ordnance Survey (OS) released a number of their mapping products for free under the brand OpenData with an attribution-only license. One part of the product suite was the Land-Form Panorama dataset, an elevation model available as either a grid (raster) or as contours. The model is based on the 1:50’000 contours from the 1980s (can’t find a ref for this on the web, but I remember reading it – so it must be true…). From the outset OS stated they would not update this dataset.

This spring (2013) OS released a new elevation model called Terrain 50 – which will be updated (annually)!

As I’ve been using Panorama for the past three years I thought it worth undertaking a quick analysis to see where the differences lay with Terrain 50, and how big they are. With two ‘perfect’ datasets this would show landscape changes of the past 20-30 years. In reality I was expecting Terrain 50 to highlight errors in the Panorama.

For this I used the open source GIS, GRASS. If you’d like to check my working you can, of course, use any GIS software of your choosing, or do it in a language like Python or R. Be warned, if you follow the steps below on a similar quantity of data you’ll need at least 4Gb of RAM.

GRASS is built up of many tools, those I’ve used I’ve included in brackets. Here are the steps I went through:

1. Imported all the tiles from both datasets into my GRASS mapset (r.in.gdal)
2. Then merged them (r.patch) into two sets: Panorama and Terrain 50
3. Subtracted the Panorama data from the Terrain 50 (r.mapcalc)
4. Exported the datasets (r.out.xyz) so I could analyse them in R, not having yet acquired the skill to use R inside GRASS…
5. Loaded the data into R (takes a couple of minutes)

   	panorama=read.table("raw/OS_elevation/panorama", header=F, sep="|")
   	terrain50=read.table("raw/OS_elevation/terrain50", header=F, sep="|")
   	

6. Plotted density curves (smoothed histograms) for both sets

   	png(file="smooth_hist.png")
   	plot(density(terrain50$V3), main="Density plot of OS Terrain 50 (black) and Panorama (red)")
   	lines(density(panorama$V3), col="red")
   	dev.off()
   	

7. Discovered there’s very little difference between them…

Terrain 50 and Panorama data for Scotland from the Ordnance Survey plotted as a density distribution.

Which isn’t a big surprise as the mountains of Scotland haven’t really changed much in the last 30 years. There is a small spike from the Terrain 50 around an elevation of 0 m AOD (Above Ordnance Datum). Most likely this is due to the inclusion of more small islands than the earlier dataset.

Fear not though, I didn’t stop there. Going back to GRASS I plotted up the difference layer I’d created in step 3, above. Generally there seemed to be little going on, but as I paid a little more attention the following differences stood out (map shows the Terrain 50 minus the Panorama):

1. The Panorama data for the west of Scotland is generally east of the Terrain 50 data by approx 50 m
   

   Terrain 50 data shifted towards the east (near Ben Nevis).
   Contains Ordnance Survey data. (c) Crown copyright and database right 2013. Data provided by EDINA, University of Edinburgh Service.

2. This hasn’t really happened for the tile(s) over Inverness
   

   Tiles near Inverness seem to match better than elsewhere…
   Contains Ordnance Survey data. (c) Crown copyright and database right 2013. Data provided by EDINA, University of Edinburgh Service.

3. There are a couple of hot spots where a big difference is observed, I’ve picked two:
   1. Polla, Loch Eriboll has grown a 500 m high round hill. I’m 99.9% certain this is an error
      

      A new mountain near Polla!
      Contains Ordnance Survey data. (c) Crown copyright and database right 2013. Data provided by EDINA, University of Edinburgh Service.

   2. Near Loch Leven a large hole has been filled in! This is probably correct as there is/was opencast workings in these parts…
      

      No longer a large, flat bottomed hole near Loch Leven.
      Contains Ordnance Survey data. (c) Crown copyright and database right 2013. Data provided by EDINA, University of Edinburgh Service.

So, important question: should you switch to using Terrain 50 instead of Panorama?

I don’t know!! As a general rule, then probably: as it is supported and should improve over time. As to which data is in the right place (west-east shift), I also don’t know.

When I’ve a free bit of time and processing power I’ll do a comparison to the NextMap data, which should give us a much better idea.

Vote for talks you want to hear at a conference!

I’ve been following the build up to the FOSS4G UK event and last week the call for presentations/papers closed.

Today I saw a link on twitter saying that voting for papers was now open. Voting? For papers? After following the link it was exactly as described! The community get to score the submitted titles and abstracts.

Time to Vote: The call for presentations for FOSS4G 2013 is now closed – but voting on the proposals subm… 2013.foss4g.org/?p=1063

— FOSS4G 2013 (@foss4g) April 15, 2013

This has benefits for everyone. All too often you don’t get a conference programme until a week or two before an event, long after you’ve paid. You’re unsure if you’ve spent your money wisely and are feeling a little nervous about this! From an organisers point of view you are making full editorial decisions about what appears at your conference with no grounded idea in what your audience might want to hear about.

Voting solves both these problems!

Other knock-on-benefits with involving the community mean greater audience buy-in, so more support and conference attendance (especially with voting before early bird registration closes); opportunity to attract targeted sponsors, backed with data; and writers having a good idea of what else will be covered to ensure their presentation stands out and ties in.

Win-win all round. Why don’t more conferences do it?

Winter special radio interview – the success of social networking

On Tuesday this week I was struggling in the office whilst there were amazing forecasts for the Scottish Highlands. Tough gig! So like many others I tweeted about my first world problems:

Wishing I’d sacked off the office and headed to the field today #scotwinter

— Michael Spencer (@MikeRSpencer) December 11, 2012

Not long afterwards I got a reply from Chris Sleight, who runs the Scottish Winter twitter feed, seconding the thought. After he’d taken a quick look at my blog I got another message from Chris asking if I’d like to appear on the BBC Radio Scotland Out of Doors show he produces. Out of Doors have (had, depending on when you read this!) a winter special on the 22^nd December. Chris and I had a quick discussion and he explained how they record much of the show outside for authenticity and if we could meet up at Crianlarich the next day we could combine the interview with a climb. Why would anyone say no?!

I was out the door at a 0645 Wednesday morning driving through a dark and freezing Scotland on the way to our meeting point in Crianlarich. There were areas of inversion, particularly Callander and Crianlarich, with partially frozen lochs and heavy frost along the roadside. Magic.

We met at 0845 and geared up for the day. Chris kindly lent me a pair of climbing axes rightly thinking they’d be more suitable for the day than my single walking axe. After a dicy few kilometres of walk in along a heavily frozen track and a bit of bog we arrived at the foot of the route – Y Gully, a grade 1 winter route with a left and right hand variant (Figure 1).

Figure 1: Y Gully left variant just to right of red line (copyright Michael Spencer)

We grabbed a quick butty and popped our crampons on, ready for the steep section ahead. Chris chose this moment to get out Hairy Mike and check the sound levels whilst he dug a quick snow pit (Figure 2), reminding me that I was there to do some work… As my first official grade 1 route – it was cracking! There was a steep, 3-4 m, section of ice early on and after that was a smooth joyful cruise up the remaining 150 m of firm snow with stunning views beyond Crianlarich behind.

Figure 2: Chris working the snow pit with Hairy Mike (copyright Michael Spencer)

Arriving at the summit we had great views all around (Figures 3 and 4) and a well deserved second lunch whilst Chris announced it was time to do some work. Having lost myself enjoying the climb it was a slight shock that I’d need to sing for my supper! Chris had prepared well on the ascent getting good ideas for interview questions so the process was easy and painless. We recorded approximately 15 minutes of material, discussing my research and the snow cover we could see from the summit. Will be interesting to see how much makes the final cut!

Figure 3: Looking North from Cruach Ardrain (copyright Michael Spencer)

Looking South West from Cruach Ardrain (copyright Michael Spencer)

I absolutely enjoyed myself and would certainly be open to another day like that!

The Out of Doors winter special will be broadcast at 0630 on 22^nd December on Radio Scotland and will be available on the BBC site for a week afterwards.

Swedish fieldwork, part 2

Following on from the previous post on my Swedish fieldwork here is the belated second and final part.

Diary

I departed the UK early on Saturday 3rd March 2012 and returned the afternoon of Wednesday 21st March 2012. I spent:

• Six productive field days
• Five days in transit
• Two office days – forced by bad weather
• Two lost days due to equipment failure
• Two days off
• Two days carrying equipment to and from site.

Fieldwork

The field site was located just to the east of the Abisko National Park, and to the south of the research station. We had two options for site access, both of which were on foot. These were using snow shoes or cross country skis. Whilst show shoes are much better for steeper terrain or manoeuvring around trees, skis are a much faster way to travel over gentle rolling terrain with tracks! Because of this each day we skied up hill to the field site which took approx 1.25 hours, with an easy 0.75 hours downhill at the end of each day. Much harder in the dark though…

A portion of my time was spent helping Rob Holden with his fieldwork which involved taking repeat measurements along transects he surveys each year. Beyond this my time was spent taking hemi photos (figure 1) as part of the project described in my original post. The aim was to combine my fieldwork with that of Rob’s to maximise our sampling and so we worked together in a safe and efficient manner.

Figure 1: Hemi photo showing sky covered by trees.

Lessons learnt

With lots of time spent travelling to site you begin to appreciate not making mistakes, or having a backup plan, so here are a few lessons learnt over the course of the expedition:

• Always have a backup fieldwork plan for each site day in case of an equipment fault or other issue with your original one
• Travelling in a straight line on skis through a dense forest is hard, even sighting with a compass. If you want to take a transect enter it into a GPS in the warm back at base!
• Skis are generally a much more energy and time efficient way to travel medium distances when compared to snow shoes, except when the terrain steepens
• A snow scooter would have saved considerable time and effort when covering the long transects enabling a larger number of sites to be surveyed. There is obviously an increase in financial cost to consider when weighing up the benefits. Travelling through forest would be much harder on a snow scooter and there would be a temptation to travel on tree free areas and only move into the forest to survey, reducing the site distance to the edge of the forest.
• The camera and tripod we used were very heavy (~5 kg), when not travelling by motorised transport this is immediately noticeable and severely affects the distance one is able to cover. The camera in particular is very expensive, which adds an unfair burden of responsibility to the user in a fieldwork setting
• Radios do not work near a dGPS repeater or rover
• Have an escalation procedure for if you don’t return from site one day…

Equipment

The equipment I used falls into three categories: data collection, clothing and transport. It’s probably important to note I haven’t received any sponsorship or endorsement from the companies detailed here.

To take the hemi photos I used a full frame Canon digital SLR fitted with a 190° fish eye lens. Whilst this is a good camera, combined with the tripod the weight was approximately 5 kg – which is a lot for fieldwork undertaken on foot! The GPSs we used were a mix of two Trimble dGPSs and a Garmin Etrex (the classic yellow one). The Trimble units are differential GPS units that use a base station connected via radio link to a rover allowing for a considerably more accurate measurement of location. The price for this increased accuracy is more gear to carry and a fair amount of effort ensuring the rover can connect to the base station. It’s no fun wading around in snow through dense forest with an aerial strapped to your back! Figure 2. The dGPS was used to locate previous transects required by Rob Holden, but was deemed not practical for the more remote hemi photo survey work. The Etrex unit provided a good compromise and Table 1 below shows a comparison made across 28 points between dGPS and the handheld Garmin unit.

Table 1: Comparison between GPS and dGPS, summary.

	GPS-dGPS
Measure	East (m)	West (m)	Elevation (m)
Minimum	-3.96	-8.68	-10.81
Maximum	6.27	5.45	7.11
Mean	1.10	0.02	-1.37
Median	1.04	0.08	-1.11
Absolute mean	1.95	2.13	3.56
Absolute median	1.66	1.53	3.20
Standard deviation	2.18	2.92	4.36

Figure 2: Rob trying to manoeuvre with a dGPS. (copyright Michael Spencer)

Given the harsh conditions present in the Arctic during the late winter clothing plays a vital role in reducing the risk posed by the cold. The best bits of kit that did this were:

• Montane Extreme smock: I can’t praise this jacket enough, the price is super low and the performance is exceptional. Worn next to the skin it dries out super fast and hence keeps you very comfortable. The jacket is a pile/pertex one with a high wicking inner overlain by a wind resistant outer. Compared to Rob, who used a traditional layered clothing system, I spent less time changing layers (only needing to adjust vent zips) and less time damp!
• Montane North Star jacket: When we were stood around for long periods, the wind blew or it was very cold I’d pull this down jacket over the pile/pertex smock mentioned above. I’m a big fan of Montane kit for three reasons; I find the athletic fit works for me, the performance is very good, it’s well made
• Buffalo mitts: Buffalo built the original pile/pertex clothing and still make top kit. Toasty warm wet or dry. The only downside with these mitts is that they’re not designed for holding, so do have the potential to wear out quickly if using ski poles a lot…
• Muck-boots Tay Sport (Arctic in US, I think…): Very warm and unlike more traditional snow boots from Sorel are fully waterproof. Drawback is that your socks fall down if you walk around a lot!
• Ron Hill Trail: Trusty pair of trousers. A fair bit thicker than the tracksters. Only downside is they don’t make them any more!

Transport was provided by backcountry skis purchased from Intersport in Kiruna. I got them on a deal with boots and to be honest I don’t know much more than that they’re metal edged, >2 m long and with some fish scales for forward propulsion! Being a skiing novice, in advance of the trip though I did take some lessons at the Midlothian Snowsports Centre. A fantastic facility if you ever have some spare time and are near the capital!

Results

After a busy summer I’m now beginning to work on the collected data with Tim Reid and Richard Essery. With a following wind we should have a paper out shortly!

In the mean time, here are some pictures from the trip:

Squeezy cheese, for champs (copyright Michael Spencer)

Skiing back to base as the sun sets (copyright Michael Spencer)

Towing the sled (copyright Michael Spencer)

Ice crystals on a frozen lake (copyright Michael Spencer)

Another late finish… (copyright Michael Spencer)

Clouds rolling (copyright Michael Spencer)

Hemi photos in action (copyright Michael Spencer)

Excuses, excuses…

So it’s been a long while since I added anything to this blog. I do have a number of posts in the pipeline but am currently waste deep in organising a conference!

For those who are interested it is the 22nd annual University of Edinburgh, School of GeoSciences GradSchool conference. Most of my time at the moment is taken up with fund-raising, so if you are interested in sponsoring the event please get in touch! It’s an excellent opportunity to hear about geoscience research from snow to oil, climate to ecology and for industry and academia to network.

I did find a few hours on sunday afternoon to head up Ben y Vrackie just north of Pitlochry. Cracking weather, apart from the half hour I spent on the top when the cloud was in…

Carn Laith from Killiecrankie/Ben y Vrackie path (copyright Michael Spencer)

Looking south from Ben y Vrackie, well: ~100 m below summit (copyright Michael Spencer)

First snow and where to look out for it

There’s talk of first season snow flying about the internet at the moment. A couple of weeks ago (30th August) there was a light dusting of fresh snow on Cairngorm and Ben Macdui.

Today Glencoe has seen a little snow too, the following picture is from the Winterhighland webcam at lunch time today (11^th September). Certainly not enough for skiing! But no denying there’s some white stuff on the ground.

http://www.winterhighland.info/cams/glencoe/

So where are good places to look for information on weather and particularly snow in Scotland? Almost without saying the UK Met Office provides mountain forecasts for the UK. Personally, in the mountains, I prefer those provided by the Mountain Weather Information Service, which is financed through sponsorship and support from government funded sportscotland. Although as both services are very good and easily accessible via a mobile device it’s sensible to check both!

If you’re heading to the hills in winter you should always see what the Scottish Avalanche Information Service has to say. www.sais.gov.uk will also automatically recognise a mobile device and provide you with just the information you need.

What about if you have a casual interest? Winterhighland has an excellent links page that pulls together webcams, forecasts and also feeds from automatic weather stations (AWS). I particularly like the climate robot pages available through the high level AWS links, suggesting here that on average 1.4 frost days can be expected on Aonach Mor during September. The hint is that lasting snow now (during September) would be the exception not the norm!