yahoo2mbox
yahoo2mbox retrieves the messages from Yahoo! Groups archive and stores them in a local file in mbox format.
| Tags | Internet |
|---|---|
| Licenses | Public Domain |
| Operating Systems | Windows Windows Windows POSIX |
| Implementation | Perl |
Recent releases
- 07 Jan 2008 12:00
Changes: This release adds a --retry option to reget any missing messages. It also adds a --debug option that allows you to save any messages the script was unable to parse as files for later analysis.
- 20 Oct 2006 04:48
Changes: The bug which could result in a corrupted MBOX file with missing blank lines between the messages was fixed. Handling of out of range message indices was fixed.
- 11 Jun 2006 09:02
Changes: Updated to work again after June 2006 changes in the Yahoo! Web site. The previous versions don't work any longer.
- 23 May 2006 23:53
Changes: A bug that resulted in corrupted output file due to the presence of extra spaces in Yahoo! Web pages was fixed. The --x-yahoo option is turned on by default, since resuming doesn't always work properly without it.
- 18 Mar 2006 14:22
Changes: Pages in both new and old format are recognized now, as country-specific Yahoo Groups sites haven't been updated yet. New --next=n and --no-post-id options have been added. Yet another error message given when the download limit is reached is recognized.
- All comments
Recent commentsBatch file to automate yahoo2mbox
Hi All
Having struggled getting my Yahoogroup downloaded in a decent amount of time using yahoo2mbox due to the amount of times my account got locked out by Yahoo, I just thought I'd share the following batch file with you that I wrote that resolved it for me, allowed me to download 60,000 messages in about 4 days without getting locked out.
The script uses 2 usernames/passwords to share the load, so if you only have one username you'll need to register another, although I just used another named profile within the same Yahoo user account as even if one gets locked out, the other seems to continue working.
Copy the entire script below into notepad and save it as runme.bat in a new folder. In the same folder you'll also need the yahoo2mail.pl file and also a file called sleep.exe which you can officially get in the Windows 2003 Resource kit, but there's a version here that will also do the job: http://www.computerhope.com/dutil.htm
Script as follows, copy between the ======
=====================================
@echo off
set /p groupname=Set Yahoo Group Name ...
set /p startat=Set message download start number (usually 0)...
set /p add=Set Number of messages to download each time (Recommend 75)...
set /p delay=Set delay time in seconds (Recommend 2)...
set /p username1=Set Username 1 ...
set /p password1=Set Password 1 ...
set /p username1=Set Username 2 ...
set /p password2=Set Password 2 ...
cls
:initial run
set /a endat=%startat%+%add%
yahoo2mbox.pl -user="%username1%" -pass="%password1%" -verbose --resume -delay=%delay% -end=%endat% "%groupname%"
sleep 90
:continue
set /a endat2=%endat%+%add%
echo %endat2%
yahoo2mbox.pl -user="%username1%" -pass="%password1%" -verbose --resume -delay=%delay% -end=%endat2% "%groupname%"
set error=%errorlevel%
echo %error%
if not %error%==0 goto retry
set endat=endat2
sleep 90
goto continue
:continue2
set /a endat2=%endat%+%add%
echo %endat2%
yahoo2mbox.pl -user="%username2%" -pass="%password2%" -verbose --resume -delay=%delay% -end=%endat2% "%groupname%"
set error=%errorlevel%
if not %error%==0 goto retry2
set endat=endat2
sleep 90
goto continue2
:retry
echo %error% >log.txt
yahoo2mbox.pl -user="%username2%" -pass="%password2%" -verbose --resume -delay=%delay% -end=%endat2% "%groupname%"
set error=%errorlevel%
set endat=endat2
if %error%==0 goto continue2
sleep 900
goto retry2
:retry2
echo %error% >log.txt
yahoo2mbox.pl -user="%username1%" -pass="%password1%" -verbose --resume -delay=%delay% -end=%endat2% "%groupname%"
set error=%errorlevel%
set endat=endat2
if %error%==0 goto continue
sleep 900
goto retry2
pause
=====================================
Once you have the batch file, the sleep.exe and the yahoo2mbox.pl file in the same location, run the batch file and it will prompt you for the required information and recommend the delay / download settings I used that seemed to work without overloading the download limits. If it does overload and lock out, it will switch to the second account, which should give time for the first one to unlock again. Once the second one locks out, it switches back to the first one again and so on.
Hope it helps someone.
Re: What about files
> Hi Is there any way to download files
> stored on a yahoo group?
>
>
If you found this tool of interest, then you might find this one helpful as well.
http://freshmeat.net/projects/grabyahoogroup/
It includes downloads for photos, files, member list, attachments and messages...
Good Luck!
WebSissy
What about files
Hi Is there any way to download files stored on a yahoo group?
Re: failed with yahoo2mbox
> yahoo2mbox failed to retrieve message
> from
> yahoogroups and give:
> <x-html> tag not found
>
> any idea how to solve this?
i hope this problem will solved soon
Re: failed with yahoo2mbox
> AFAICS yahoo has changed the ouput of
> view source
> messages.
>
> try searching for <!-- start content
> include
> -->
> instead of <x-html>. Then you have
> to deal
> with
> the multiple <tt> tags, that are
> used to
> contain
> different parts of the message. eg the
> header is
> one tt, the body is another & if the
> message is
> multipart/mime there will be heaps of
> <tt>stuff
> here</tt><br> & it all needs
> to be
> converted.
>
> please im/email me if you want a sample
>
Have somebody been able to solve this "x-html tag not found" error message?
I use yahoo2mbox 0.14 and the problem still occured.