The downloader is broken, and there is no prospect of it ever working again.
Tuesday 5 December 2017
It's Dead
Regrettable, but inevitable. The AFL's replay infrastructure has been completely changed as of late 2017.
The downloader is broken, and there is no prospect of it ever working again.
The downloader is broken, and there is no prospect of it ever working again.
Saturday 6 June 2015
Video File Downloader, 2015-2017
At the beginning of 2013 the friendly Telstra engineers threw away the simple filing pattern that was once in place, with the result that the video links no longer showed any sensible pattern at all. It was possible to discover them after the replays went live but it was apparently not possible to predict anything ahead of time. So for two years the community had to be served by altruistic contributors who posted the game links, one by one, as they went up.
The situation became worse in 2015 when Telstra decided to put all their replays past and present behind a paywall. This reduced the number of people who could discover the links in the first place (at least by any of the methods covered earlier in this series) to those who had paid for an AFL Live Pass.
But a solution to the problem soon came to light, and Sandbelt is indebted to Zappy_32 who in April posted a very useful algorithm on BigFooty's Match Replay Thread. The algorithm (which I won't go so far as to repeat here) is so regular that it invites someone to write a program to automate it.
And so, Sandbelt Software presents a replay downloader for Windows that gives access to everything from Round 1 2010 onwards. It looks like this:
There is no installer: just download the zip file, extract AFLReplays.exe to somewhere convenient and run it. Be aware though that the AFL servers are geoblocked to Australia and NZ, so if you live elsewhere you will need a VPN.
Other OSs
AFLReplays is a Windows program, based on the .NET framework. As such it has been run successfully in Linux under mono, installed like this. By extension it will also run in MacOS, though Sandbelt Software offers no guides or technical assistance in this respect.
Caveats
This program, as goes without saying, come without warranty, implied support or express permissions other than that you can do what you like in private.Monday 14 May 2012
Update for the 2012 season
This post continues a series starting at http://sandbelt-sandpit.blogspot.com/2011/09/how-to-download-afl-videos.html. The series was mostly composed in 2011 when there was no guarantee of what might happen in the next year. Now it is a third of the way into the 2012 season and it is clear that the engineers at Telstra are happy to continue with their previous practices, at least as far as locating the game videos is concerned, and this is a good thing.
All games from 2010 onwards are stored on the Telstra servers as simple mp4 files that are easy to download once anybody has a link to them. The AFL website used to make this simple to do, but in recent years all the file links have been hidden behind a wall of javascript (the BigPond AFL videos website is just as bad).
The 7 September 2011 post went into some detail about how you can discover the file links and ended up with a pattern that could predict the url for any quarter of any game in 2011. I am now fairly confident about how to make predictions for the current year.
Q1
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_1st_qr_full_2M.mp4
Q2
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_2nd_qr_full_2M.mp4
Q3
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_3rd_qr_full_2M.mp4
Q4
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_4th_qr_full_2M.mp4
To make real links, replace nn (in both places) with the round number, hhh with the home team name and aaa with the away team name. Round numbers are like 01, 02, 03 and so on. Team names are adel, bl, carl, coll, ess, fre, geel, gcfc, gws, haw, melb, nmfc, port, rich, stk, syd, wce or wb.
One minor glitch:
Gold Coast has to be written gdfc in (and only in) rounds 02 and 03. I assume that this is somebody's carelessness.
For Bombers Fans:
Sandbelt has used this pattern to predict where the Essendon videos will be for the whole of all the current home-and-away season. Left-click a table cell to view, right-click (or, I hear, Apple+click) to download.
Downloads do not count towards your quota if you are a BigPond customer.
Videos typically appear a few hours after the game ends.
To recap:
All games from 2010 onwards are stored on the Telstra servers as simple mp4 files that are easy to download once anybody has a link to them. The AFL website used to make this simple to do, but in recent years all the file links have been hidden behind a wall of javascript (the BigPond AFL videos website is just as bad).
The 7 September 2011 post went into some detail about how you can discover the file links and ended up with a pattern that could predict the url for any quarter of any game in 2011. I am now fairly confident about how to make predictions for the current year.
The Pattern:
The links for 2012 look like this, so far:Q1
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_1st_qr_full_2M.mp4
Q2
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_2nd_qr_full_2M.mp4
Q3
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_3rd_qr_full_2M.mp4
Q4
http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2012/ON/iVideo/Premiership/RDnn/AFL12_rdnn_hhh_vs_aaa_4th_qr_full_2M.mp4
To make real links, replace nn (in both places) with the round number, hhh with the home team name and aaa with the away team name. Round numbers are like 01, 02, 03 and so on. Team names are adel, bl, carl, coll, ess, fre, geel, gcfc, gws, haw, melb, nmfc, port, rich, stk, syd, wce or wb.
One minor glitch:
Gold Coast has to be written gdfc in (and only in) rounds 02 and 03. I assume that this is somebody's carelessness.
For Bombers Fans:
Sandbelt has used this pattern to predict where the Essendon videos will be for the whole of all the current home-and-away season. Left-click a table cell to view, right-click (or, I hear, Apple+click) to download.
|
|
Downloads do not count towards your quota if you are a BigPond customer.
Videos typically appear a few hours after the game ends.
Wednesday 7 September 2011
Getting the AFL Videos from 2008 and 2009
This post continues a series starting at http://sandbelt-sandpit.blogspot.com/2011/09/how-to-download-afl-videos.html. The first post made the following remark:
"[AFL games] from 2008 and 2009 are available only as Windows Media rtsp streams. The quality is more-or-less the same: 672x544 resolution at a video rate of 1.99 mbps, though it turns out to be harder to determine the link address and harder again to download the streams to disk files."
So there are two problems: first we need the urls of the media streams we want, and then we need some means of recording them. But first some background knowledge.
rtsp?
This is Real Time Streaming Protocol, and all you need to know is that it is is a network control protocol designed for use in entertainment and communications systems, that some media players can handle it directly and some can't, and if they can't they may negotiate an alternative distribution over the very common http protocol.
record media streams?
There are many providers that supply video on a watch-now basis, and they do it by packaging in stream form. These streams are not intended to be recorded, and this means that there are several commercial packages that do just that, in exchange for money. I have tried a few without much liking any of them, and nowadays I try to use VLC Media Player which is very powerful (if sometimes hard to drive) and also free. And also works on ten versions of Linux as well as OSX, iOS, Android and of course Windows. Everybody ought to have it.
VLC is supposed to handle rtsp streams but it doesn't do it very well, at least in version 1.1 11, and is much happier being fed over http. Fortunately this is possible.
How to discover the urls,
and who doesn't need to know.
Second part first: you only need to read the following if (a) you want to, or (b) you want a game that Essendon wasn't in. Bombers fans can afford drop right down to the How to record a stream section because all the links they will need can be found at the end of this post.
But if you are still here, to continue, the DownloadHelper/FireFox solution used in my Post number 2 will not work with stream videos and you will have to resort to eavesdropping on your browser's communications with the outside world. The application I use for this is URL Snooper, a program written to help users locate the urls of audio and video files so that they can be recorded. Get this program from http://www.donationcoder.com/Software/Mouser/urlsnooper/index.html. It is donationware, which means that it is available for free but you can pay if your own sense of justice tells you to. It runs only on Windows, and (sorry) I don't know about anything else.
So, assuming you have a Windows machine, install URL Snooper, turn it on and click the Sniff Network button. The Keyword Filter box is to reduce clutter and it would be good at this point to type a team name in there.
As previously, go to BigPond AFL Videos and use the fancy navigator to select ON DEMAND, Essendon TV (or some other club if that is what you really want) and Matches. Set the quality as high as possible in the web player. Pick a match and play it.
If you can read this image you will observe a whole list of rstp stream requests, including the one we are playing (rtsp://bptvvod.ngcdn.telstra.com/wh_ivideo/AFL/ONDEMAND/2009/ON/iVideo/Premiership/RD13/NV_Rd13_EssendonVCarlton.wmv?WMBitrate=2621440). Further down you see the same game appearing as a WindowsMedia http file, and its URL is otherwise identical except for a changed server name (now fli-cdn220-is-1.se.bptvvod.ngcdn.telstra.com).
At this point it helps to know that BigPond has a lot of servers and uses a sort of alias system to direct requests to them. The bptvvod.ngcdn.telstra.com server name seems to be an umbrella that resolves to all the other alternatives, or at least it has so far. We might then guess that we could ignore the server name and just change the rtsp:// part of the stream URL to http://, and sure enough the following procedure has proven to be reliable:
There have been changes to VLC since all I first wrote all of the above and it seems to have developed a bug that causes it to crash. But you can always just use an older version, such as 1.1.11 which you can download from HERE. The goodish news is that VLC is a portable app that does not need to be actually installed, so you can download the old version as a zip file, unpack it somewhere (even to a USB stick) and then just run its main program (vlc.exe) from your file manager.
2009
(*) Low quality
2008
The final post in this series, when I write it, will be about how to record the videos from 2007. This will be getting very much into the land of diminishing returns since their quality is so low, but never mind, completeness is a virtue all in itself.
"[AFL games] from 2008 and 2009 are available only as Windows Media rtsp streams. The quality is more-or-less the same: 672x544 resolution at a video rate of 1.99 mbps, though it turns out to be harder to determine the link address and harder again to download the streams to disk files."
So there are two problems: first we need the urls of the media streams we want, and then we need some means of recording them. But first some background knowledge.
rtsp?
This is Real Time Streaming Protocol, and all you need to know is that it is is a network control protocol designed for use in entertainment and communications systems, that some media players can handle it directly and some can't, and if they can't they may negotiate an alternative distribution over the very common http protocol.
record media streams?
There are many providers that supply video on a watch-now basis, and they do it by packaging in stream form. These streams are not intended to be recorded, and this means that there are several commercial packages that do just that, in exchange for money. I have tried a few without much liking any of them, and nowadays I try to use VLC Media Player which is very powerful (if sometimes hard to drive) and also free. And also works on ten versions of Linux as well as OSX, iOS, Android and of course Windows. Everybody ought to have it.
VLC is supposed to handle rtsp streams but it doesn't do it very well, at least in version 1.1 11, and is much happier being fed over http. Fortunately this is possible.
How to discover the urls,
and who doesn't need to know.
Second part first: you only need to read the following if (a) you want to, or (b) you want a game that Essendon wasn't in. Bombers fans can afford drop right down to the How to record a stream section because all the links they will need can be found at the end of this post.
But if you are still here, to continue, the DownloadHelper/FireFox solution used in my Post number 2 will not work with stream videos and you will have to resort to eavesdropping on your browser's communications with the outside world. The application I use for this is URL Snooper, a program written to help users locate the urls of audio and video files so that they can be recorded. Get this program from http://www.donationcoder.com/Software/Mouser/urlsnooper/index.html. It is donationware, which means that it is available for free but you can pay if your own sense of justice tells you to. It runs only on Windows, and (sorry) I don't know about anything else.
So, assuming you have a Windows machine, install URL Snooper, turn it on and click the Sniff Network button. The Keyword Filter box is to reduce clutter and it would be good at this point to type a team name in there.
As previously, go to BigPond AFL Videos and use the fancy navigator to select ON DEMAND, Essendon TV (or some other club if that is what you really want) and Matches. Set the quality as high as possible in the web player. Pick a match and play it.
URL Snooper will have picked up all the media links that the browser made requests for:
At this point it helps to know that BigPond has a lot of servers and uses a sort of alias system to direct requests to them. The bptvvod.ngcdn.telstra.com server name seems to be an umbrella that resolves to all the other alternatives, or at least it has so far. We might then guess that we could ignore the server name and just change the rtsp:// part of the stream URL to http://, and sure enough the following procedure has proven to be reliable:
- Play the game, snoop its rstp URL
- Edit this url so it starts with http:// instead of rstp://.
How to record the stream.
Here is where we use VLC. Get it from http://www.videolan.org/vlc/ (if you don't have it already) and fire it up.
Click the Media menu and select Open Network Stream...
Paste in your edited (http://) game URL and click the little down arrow to the right of the Play button:
Write in a destination file name ending in .mp4. Optionally check the Display the output box.
Click Start. You are up and running.
The recording runs in real time, so a 3-hour video will take 3 hours to record. On the other hand if you have a good internet connection you can start several instances of VLC and record several games at once.
Update: A problem with VLC 2.0?
For Bombers Fans
Friendly sandbelt has constructed lists of URLs of essendon games for the two years that this video format was current for. Some browsers (for me, only Internet Explorer 9) allow you to check them out directly by clicking the links. But you can always right-click the link, choose Copy, and then paste into VLC to make a recording.
2009
|
2008
|
The final post in this series, when I write it, will be about how to record the videos from 2007. This will be getting very much into the land of diminishing returns since their quality is so low, but never mind, completeness is a virtue all in itself.
Getting the AFL Videos from 2010 and 2011
All games from 2010 and 2011 are stored on the BigPond servers as simple mp4 files and would be easy to download if anybody had a link to them. The AFL website used to make this a simple problem, but the world is full of spoilsports and nowadays all the file links are hidden behind a wall of javascript. The BigPond AFL videos website is no friendlier.
But it is not hard to discover a video link if you use FireFox.
Using FireFox and Download Helper
You can install DownloadHelper from the FireFox Add-ons manager or else by going directly to http://www.downloadhelper.net/. Both FireFox and DownloadHelper are (of course) free.
Then go to BigPond AFL Videos and use the fancy navigator to select ON DEMAND, Essendon TV (OK, some other club if that is what you really want) and Matches. Pick a match and it will start to play in an embedded Flash machine. Do not omit to set the quality as high as possible (there is a control for that under the video, on the right).
Now you will notice that the the DownloadHelper icon at the top of your browser has started to animate, and if you click the little down arrow next to the icon you will see the name of the file you are playing.
Wave your mouse over this name and one of your options is to download the file. Another option is to copy the file's URL, and you can do this if you prefer to download later or if (like me) you plan to share the link with somebody else.
That's about it. As far as I know, any download from a BigPond server is unmetered for anybody who uses BigPond for internet access. I have never bothered to confirm this because I am signed up with iinet and it comes of my quota in any case. Perhaps somebody might comment.
Patterns.
After you have discovered a few of these link addresses you start to see that they follow a predictable pattern. This is because the Telstra engineers use scripts to create the video files in the first place, and it has the benefit of making it easy for people like myself to use scripts to download them. It also means that you can predict the urls for any game from a few simple rules.
For most of 2010 and all of 2011 the pattern has been like this:
Q1 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_1st_qr_full_2M.mp4
Q2 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_2nd_qr_full_2M.mp4
Q3 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_3rd_qr_full_2M.mp4
Q4 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_4th_qr_full_2M.mp4
To make real links, replace nn (in both places) with the round number, hhh with the home team name and aaa with the away team name. Round numbers are 01, 02, 03 and so on. Team names are adel, bl, carl, coll, ess, fre, geel, gcfc, haw, melb, nmfc, port, rich, stk, syd, wce or wb.
The last half of 2010 used the same pattern except that 2011 used to be 2010 and AFL11 was AFL10. You can see the pattern working in the tables underneath.
For Bombers Fans
Friendly sandbelt has a list of all Essendon matches from 2010 and 2011 (corrected 9/9/2011). If you want anything, just right-click a link:
2011
Mind you. I'm guessing about the final, it hasn't happened yet as I write this.
2010
The next post will go into what you have to do to download the media streams from 2009 and 2008. I will eventually even cover 2007, because even though the quality is lousy there is some good historical stuff there, and I admit I got a kick out of seeing Mark Johnson with long white hair.
But it is not hard to discover a video link if you use FireFox.
Using FireFox and Download Helper
You can install DownloadHelper from the FireFox Add-ons manager or else by going directly to http://www.downloadhelper.net/. Both FireFox and DownloadHelper are (of course) free.
Then go to BigPond AFL Videos and use the fancy navigator to select ON DEMAND, Essendon TV (OK, some other club if that is what you really want) and Matches. Pick a match and it will start to play in an embedded Flash machine. Do not omit to set the quality as high as possible (there is a control for that under the video, on the right).
Now you will notice that the the DownloadHelper icon at the top of your browser has started to animate, and if you click the little down arrow next to the icon you will see the name of the file you are playing.
Wave your mouse over this name and one of your options is to download the file. Another option is to copy the file's URL, and you can do this if you prefer to download later or if (like me) you plan to share the link with somebody else.
That's about it. As far as I know, any download from a BigPond server is unmetered for anybody who uses BigPond for internet access. I have never bothered to confirm this because I am signed up with iinet and it comes of my quota in any case. Perhaps somebody might comment.
Patterns.
After you have discovered a few of these link addresses you start to see that they follow a predictable pattern. This is because the Telstra engineers use scripts to create the video files in the first place, and it has the benefit of making it easy for people like myself to use scripts to download them. It also means that you can predict the urls for any game from a few simple rules.
For most of 2010 and all of 2011 the pattern has been like this:
Q1 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_1st_qr_full_2M.mp4
Q2 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_2nd_qr_full_2M.mp4
Q3 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_3rd_qr_full_2M.mp4
Q4 http://bptvpd.ngcdn.telstra.com/pd_afl0/OnDemand/2011/ON/iVideo/Premiership/RDnn/AFL11_rdnn_hhh_vs_aaa_4th_qr_full_2M.mp4
To make real links, replace nn (in both places) with the round number, hhh with the home team name and aaa with the away team name. Round numbers are 01, 02, 03 and so on. Team names are adel, bl, carl, coll, ess, fre, geel, gcfc, haw, melb, nmfc, port, rich, stk, syd, wce or wb.
The last half of 2010 used the same pattern except that 2011 used to be 2010 and AFL11 was AFL10. You can see the pattern working in the tables underneath.
For Bombers Fans
Friendly sandbelt has a list of all Essendon matches from 2010 and 2011 (corrected 9/9/2011). If you want anything, just right-click a link:
2011
|
|
2010
|
|
The next post will go into what you have to do to download the media streams from 2009 and 2008. I will eventually even cover 2007, because even though the quality is lousy there is some good historical stuff there, and I admit I got a kick out of seeing Mark Johnson with long white hair.
Subscribe to:
Posts (Atom)