Please archive content from the forums at WotC!

Industry news, rumoured and announced new products, and discussion of games company policies.
User avatar
Giant Space Hamster
Initiate
Posts: 337
Joined: Tue Jun 07, 2011 8:55 pm
Gender: male
Location: London, UK
Contact:

Please archive content from the forums at WotC!

Post by Giant Space Hamster » Sat Sep 26, 2015 1:48 pm

The Piazza just made an open invitation to all Dungeons & Dragons fans. Now we are making a call to arms. Please help copy content from the various forums for campaign settings and rules at WotC to the forums for the same campaign settings and rules at The Piazza.

Topics where people can discuss archiving content for specific D&D campaign settings (deciding what to save, etc): Topics where people can discuss archiving content for other things (deciding what to save, etc): How to archive WotC forum content at The Piazza:
  1. Register for an account at The Piazza (if you do not already have an account here),
  2. Use one browser tab/window to navigate to a specific forum at the WotC forums that you would like to save content from,
  3. Use a second browser tab/window to navigate to the specific forum at The Piazza forums that you would like to move the content to,
  4. Copy the thread title from the forum at WotC,
  5. Use the "New Topic" button at the forum at The Piazza and paste in the topic title,
  6. Copy the first post from the WotC forum:
    • If you have an account at WotC, you can use the quote button, copy the post and then paste it into your topic here
    • If you do not have an account at WotC, you can highlight the text of the post, copy it and then paste it into your topic here
  7. You need to repeat the above two steps for every reply in the topic*.
  8. Check the WotC forum posts for links and special formatting and add them back in here, if you can**.
* = If you copy each post separately, that will give the forum admins the ability to "give the post back" to the person who made it, if they want to get their post back If you copy multiple WotC posts into a single Piazza post, then the post will stay as your post.
** = If you have trouble with formatting, you can ask for help here.

This post may be edited, to add in more information, as other specific archive attempts get made.
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
This account is only used for forum administration. For general contact, please use my personal user account.
Administrator at The Piazza. My moderator voice is green.

User avatar
Canageek
Hobgoblin
Posts: 45
Joined: Thu Sep 04, 2008 3:41 am
Gender: male
Location: Canada
Contact:

Re: Please archive content from the forums at WotC!

Post by Canageek » Wed Sep 30, 2015 6:25 am

This seems like a far to large undertaking to pull off in time; Has anyone looked into an automated solution that we could untangle later?

User avatar
religon
Bugbear
Posts: 134
Joined: Mon Nov 26, 2012 8:09 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by religon » Wed Sep 30, 2015 3:06 pm

I think ENWorld has an automated tool (PC's but not Macs). It might be easier to save to their 'lifeboat' boards.

User avatar
Big Mac
Giant Space Hamster
Posts: 22733
Joined: Sun Jun 15, 2008 3:52 pm
Gender: male
Location: London UK
Contact:

Re: Please archive content from the forums at WotC!

Post by Big Mac » Thu Oct 01, 2015 12:09 am

Canageek wrote:This seems like a far to large undertaking to pull off in time; Has anyone looked into an automated solution that we could untangle later?
I don't have the ability to make that sort of thing, but if you can look for someone who might be able to help, that would be great.

Meanwhile, I have started to rescue Spelljammer topics and will move onto another campaign setting, if I get that finished.

I've not decided which one to go for, but given that The Piazza has the world's most active Mystara community, I think that the Mystara fans should have the ability to take care of that setting.

Archiving all of Forgotten Realms and/or Eberron will probably be impossible for me, in the timescale. I might go for a smaller campaign setting, just so that I can have a chance to archive all of it.
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.

User avatar
Big Mac
Giant Space Hamster
Posts: 22733
Joined: Sun Jun 15, 2008 3:52 pm
Gender: male
Location: London UK
Contact:

Re: Please archive content from the forums at WotC!

Post by Big Mac » Thu Oct 01, 2015 12:15 am

religon wrote:I think ENWorld has an automated tool (PC's but not Macs). It might be easier to save to their 'lifeboat' boards.
It is only a semi-automated too. You still have to do some of the work.

I believe that MerricB's tool generates BB Code that can be posted into any forum that supports BB Code (which is now pretty much anywhere except Wizards of the Coast.

You are free to post things in EN World, or anywhere else, but considering that The Piazza split off from the Wizards of the Coast forum, back in 2008, we would love it if the topics that many of the forum members here posted before The Piazza existed could be brought back to our community.
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.

User avatar
willpell
Black Dragon
Posts: 2896
Joined: Wed Sep 23, 2015 9:10 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by willpell » Sun Oct 04, 2015 10:33 pm

"Just because everyone shouldn't be like me, doesn't mean no-one should. Sometimes it takes a loud, angry asshole, who's full of himself, to say things that kinder, gentler people wouldn't say."
--The Amazing Atheist

Salvaged links:
http://www.asmor.com/scripts/4eMagicIte ... easure.php

D&D character:
viewtopic.php?f=75&t=17559&p=195819#p195819

User avatar
Big Mac
Giant Space Hamster
Posts: 22733
Joined: Sun Jun 15, 2008 3:52 pm
Gender: male
Location: London UK
Contact:

Re: Please archive content from the forums at WotC!

Post by Big Mac » Wed Oct 07, 2015 7:32 pm

No idea, I'm afraid. I know I have not done them, as I am still working on archiving Spelljammer topics.

If you use the forum search system, you could search for the names of the topics, to see if they are here. Maybe you could start archiving one or more of these topics yourself. :)
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.

User avatar
willpell
Black Dragon
Posts: 2896
Joined: Wed Sep 23, 2015 9:10 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by willpell » Thu Oct 08, 2015 1:02 am

Big Mac wrote:If you use the forum search system, you could search for the names of the topics, to see if they are here. Maybe you could start archiving one or more of these topics yourself. :)
I can see about it, but I'm not going to have a lot of time this month. Had Wizards announced an October closure back in January, I would have had something to do with myself when I was content to be bored out of my skull, slowly consuming my savings account. Now I actually have to work, and a month is nowhere close to long enough for this amount of archiving.
"Just because everyone shouldn't be like me, doesn't mean no-one should. Sometimes it takes a loud, angry asshole, who's full of himself, to say things that kinder, gentler people wouldn't say."
--The Amazing Atheist

Salvaged links:
http://www.asmor.com/scripts/4eMagicIte ... easure.php

D&D character:
viewtopic.php?f=75&t=17559&p=195819#p195819

User avatar
Big Mac
Giant Space Hamster
Posts: 22733
Joined: Sun Jun 15, 2008 3:52 pm
Gender: male
Location: London UK
Contact:

Re: Please archive content from the forums at WotC!

Post by Big Mac » Thu Oct 08, 2015 3:11 am

willpell wrote:
Big Mac wrote:If you use the forum search system, you could search for the names of the topics, to see if they are here. Maybe you could start archiving one or more of these topics yourself. :)
I can see about it, but I'm not going to have a lot of time this month. Had Wizards announced an October closure back in January, I would have had something to do with myself when I was content to be bored out of my skull, slowly consuming my savings account. Now I actually have to work, and a month is nowhere close to long enough for this amount of archiving.
I have the same issue. I don't have much time. And I'm having connectivity problems and I'm not well. (I would be asleep right now if I hadn't been kept up by coughing.)

So I'm going to try to finish archiving Spelljammer before October 29th and then see if I can move onto trying to save some of something else.

I put the word out, as much as I could a couple of weeks earlier, but I've not got the time to co-ordinate things and also to save things. So I think we are all going to need to save what we can and if some stuff is lost, then it is lost.

But if you know of any other people, who might have more time than you, perhaps you can see if they have the time to save these topics that you think are worth saving. I've pretty much told everyone I know, but I'm sure there are some D&D fans out there who are totally unaware the WotC forums are due to be shut down in just over 20 days. :)
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.

User avatar
Sturm
Green Dragon
Posts: 4199
Joined: Sat Feb 12, 2011 6:26 pm
Gender: male
Location: Genoa, Italy
Contact:

Re: Please archive content from the forums at WotC!

Post by Sturm » Thu Oct 08, 2015 8:58 am

The best would be to save the whole forum with an offline viewer. I'm trying something of the sort with the Other Worlds subforum now. I have no idea however if that's possible and what kind of size the final result will have.
Also, even if I manage to do that, it will be difficult or impossible to give access to it to others.
It's just another time in which Wizards has shown scarce consideration toward its customers. They could have simply close the forums and keep them online for reference, it would have not been such a big effort to do that.
Anyway I think all the Mystara material has already been archived in the Vaults, so I'm not concerned about it. Do not know what's the status of other world's material however.

User avatar
willpell
Black Dragon
Posts: 2896
Joined: Wed Sep 23, 2015 9:10 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by willpell » Thu Oct 08, 2015 9:08 pm

Sturm wrote:The best would be to save the whole forum with an offline viewer. I'm trying something of the sort with the Other Worlds subforum now. I have no idea however if that's possible and what kind of size the final result will have.
Also, even if I manage to do that, it will be difficult or impossible to give access to it to others.
It's just another time in which Wizards has shown scarce consideration toward its customers. They could have simply close the forums and keep them online for reference, it would have not been such a big effort to do that.
Anyway I think all the Mystara material has already been archived in the Vaults, so I'm not concerned about it. Do not know what's the status of other world's material however.
Does anybody on this board intend to save any of the MTG material? A lot of it goes obsolete once a format rotates, but there's also some really cool creative stuff that's difficult to so much as locate.
"Just because everyone shouldn't be like me, doesn't mean no-one should. Sometimes it takes a loud, angry asshole, who's full of himself, to say things that kinder, gentler people wouldn't say."
--The Amazing Atheist

Salvaged links:
http://www.asmor.com/scripts/4eMagicIte ... easure.php

D&D character:
viewtopic.php?f=75&t=17559&p=195819#p195819

User avatar
Gravesgard
Goblin
Posts: 10
Joined: Fri Oct 02, 2015 11:38 pm
Gender: male
Contact:

Re: Please archive content from the forums at WotC!

Post by Gravesgard » Fri Oct 09, 2015 6:03 am

I've been saving a ton of ravenloft pages, html and files. So they can be accessed offline. I'll post them up when I have some more free time.

User avatar
Giant Space Hamster
Initiate
Posts: 337
Joined: Tue Jun 07, 2011 8:55 pm
Gender: male
Location: London, UK
Contact:

Re: Please archive content from the forums at WotC!

Post by Giant Space Hamster » Fri Oct 09, 2015 7:20 am

willpell wrote:Does anybody on this board intend to save any of the MTG material? A lot of it goes obsolete once a format rotates, but there's also some really cool creative stuff that's difficult to so much as locate.
I can not speak for Ashtagon, but this forum has never been a CCG forum. It's core has been to be a D&D forum, for campaign settings and rules.

The Piazza has grown well beyond it's original remit, but it has grown to cover non D&D campaign settings and rules - not to cover CCGs and other forms of gaming. So I do not think there a fanbase here that could facilitate a rescue attempt in the timescale. (If we had been prewarned of the WotC forum shutdown several years back, Ashtagon may have considered making a home for M:tG fans, and we may have been in a position to mobilise a large number of CCG fans. But that didn't happen, so I personally do not think that there is anyone here who can take on the vast workload of saving that content.)

If you wish to save M:tG material, I would strongly suggest you sign up to an existing M:tG forum, where they do not have any sort of need to save the D&D side of the WotC forums, and where they can focus 100 percent on saving the M:tG forum content. You have less than 20 days to organise this.

Good luck!
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
This account is only used for forum administration. For general contact, please use my personal user account.
Administrator at The Piazza. My moderator voice is green.

User avatar
Angel Tarragon
Dawn Dragon
Posts: 8321
Joined: Sun Jun 06, 2010 2:39 am
Gender: male
Location: Phoenix, AZ
Contact:

Re: Please archive content from the forums at WotC!

Post by Angel Tarragon » Fri Oct 09, 2015 5:41 pm

Gravesgard wrote:I've been saving a ton of ravenloft pages, html and files. So they can be accessed offline. I'll post them up when I have some more free time.
I have archived some of the older threads, I intend do more today, maybe even get it completely done.

thorr-kan
Ogre
Posts: 208
Joined: Thu Sep 16, 2010 7:40 pm

Re: Please archive content from the forums at WotC!

Post by thorr-kan » Fri Oct 09, 2015 6:50 pm

There is a concerted effort to archive the forums and the old d20 content being made through the www.giantitp.com forums.

User avatar
enderxenocide0
Stone Giant
Posts: 700
Joined: Thu Dec 18, 2014 1:44 am
Gender: male

Re: Please archive content from the forums at WotC!

Post by enderxenocide0 » Mon Oct 12, 2015 12:56 pm

I pooled together the collective knowledge of my friends and I in the software dev business and we're currently testing an application that lets us go through a forum and save all the content to a local hard drive, making it fully browsable offline. Once we're sure it works properly in the next few days, we'll share it with the community so anyone can back up a forum in full. It won't have search capabilities unless someone makes one for it, since Wizards is unlikely to release their database to us. What you do with the content once it's stored locally is up to you. If nothing else, it gives us an infinite amount of time to move content to new forums. We would be no longer constrained by time. I'll keep you all posted on the progress.

User avatar
willpell
Black Dragon
Posts: 2896
Joined: Wed Sep 23, 2015 9:10 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by willpell » Mon Oct 12, 2015 3:24 pm

enderxenocide0 wrote:I pooled together the collective knowledge of my friends and I in the software dev business and we're currently testing an application that lets us go through a forum and save all the content to a local hard drive, making it fully browsable offline. Once we're sure it works properly in the next few days, we'll share it with the community so anyone can back up a forum in full. It won't have search capabilities unless someone makes one for it, since Wizards is unlikely to release their database to us. What you do with the content once it's stored locally is up to you. If nothing else, it gives us an infinite amount of time to move content to new forums. We would be no longer constrained by time. I'll keep you all posted on the progress.
This sounds great. Sadly I have no home computer. Does anyone on this board care enough about Magic to take care of those forums in this way?

User avatar
enderxenocide0
Stone Giant
Posts: 700
Joined: Thu Dec 18, 2014 1:44 am
Gender: male

Re: Please archive content from the forums at WotC!

Post by enderxenocide0 » Mon Oct 12, 2015 3:43 pm

If I get the time, I'll be happy to do it with any and every WotC forum (but don't necessarily rely on me as my priority will be Eberron first and I have limited time in the coming weeks).

User avatar
Big Mac
Giant Space Hamster
Posts: 22733
Joined: Sun Jun 15, 2008 3:52 pm
Gender: male
Location: London UK
Contact:

Re: Please archive content from the forums at WotC!

Post by Big Mac » Wed Oct 14, 2015 12:49 am

thorr-kan wrote:There is a concerted effort to archive the forums and the old d20 content being made through the http://www.giantitp.com forums.
Please would you post the link to the place/topic where this is being discussed/organised.
enderxenocide0 wrote:I pooled together the collective knowledge of my friends and I in the software dev business and we're currently testing an application that lets us go through a forum and save all the content to a local hard drive, making it fully browsable offline. Once we're sure it works properly in the next few days, we'll share it with the community so anyone can back up a forum in full. It won't have search capabilities unless someone makes one for it, since Wizards is unlikely to release their database to us. What you do with the content once it's stored locally is up to you. If nothing else, it gives us an infinite amount of time to move content to new forums. We would be no longer constrained by time. I'll keep you all posted on the progress.
That's great.

I also have a friend on Facebook who is attempting the same sort of thing. He said he is trying to make something called MAFF files. Obviously both of you (and anyone else attempting it) only have a fixed amount of time to catch things before WotC pull down the original source.

I would suggest you do testing on a long multipage topic (like the Commoner Campaign topic) to see if your app can cope with something like that that is a bit weird.

I'm guessing that your app is probably going to be PC based (which will mean that I probably won't be able to use it) but perhaps you can work with the data afterwards and convert that into some useable form.

If you think you could make a fully hyperlinked browsable archive, perhaps we could talk to Ashtagon about the possibility of creating an archive on The Piazza. I would like to be able to import topics into The Piazza somehow (as I have been doing with the Spelljammer topics) as that would allow WotC conversations to be reactivated.

(I've been trying to do one Piazza post per WotC post, to give me the theoretical ability to give a post back to the original poster...assuming I can confirm that it is the same person. In theory that could mean that we could capture all of Keith Baker's Eberron topics/posts, with your app, eventually get them transferred to the Eberron forum at The Piazza and then eventually get them transferred over to his Piazza account, so that he could edit and update things, if necessary. But that would probably be something that got finished a year or more after the WotC forum was gone.)
willpell wrote:Does anyone on this board care enough about Magic to take care of those forums in this way?
I would strongly advise you to get in touch with M:tG communities as soon as possible. There are only 15 days left. You could perhaps act as a middleman, helping them to get a copy of Ender's app, but don't wait for him. If he gets into technical difficulties and runs out of time, they might not have time to come up with their own solution. The sooner you can get the word out, the more chance there is that a M:tG fan can come up with an automated solution to save their content and get it hosted on a M:tG or CCG fansite.

And if they are already ahead of us, perhaps you could return here with news of a fully working archive system. :)
enderxenocide0 wrote:If I get the time, I'll be happy to do it with any and every WotC forum (but don't necessarily rely on me as my priority will be Eberron first and I have limited time in the coming weeks).
If you can save the entire Eberron forum, that would be awesome. It is the second largest campaign setting forum at the WotC forums, and it would be impossible to grab it using the copy and paste technique I've used to save about 20 percent of the Spelljammer topics. ;(

You are doing a great thing. Thank you! If you are able to go beyond Eberron and save the rest of the campaign setting area...and then the entire D&D area...that would be a major contribution to D&D fandom. But I would expect you to look after Eberron first. :)

Keep up the good work. And good luck!
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.

User avatar
enderxenocide0
Stone Giant
Posts: 700
Joined: Thu Dec 18, 2014 1:44 am
Gender: male

Re: Please archive content from the forums at WotC!

Post by enderxenocide0 » Wed Oct 14, 2015 4:31 am

Big Mac wrote:I'm guessing that your app is probably going to be PC based (which will mean that I probably won't be able to use it) but perhaps you can work with the data afterwards and convert that into some useable form.
It's actually written in Java, so it should run on anything with the JVM.

User avatar
Big Mac
Giant Space Hamster
Posts: 22733
Joined: Sun Jun 15, 2008 3:52 pm
Gender: male
Location: London UK
Contact:

Re: Please archive content from the forums at WotC!

Post by Big Mac » Thu Oct 15, 2015 12:36 pm

enderxenocide0 wrote:
Big Mac wrote:I'm guessing that your app is probably going to be PC based (which will mean that I probably won't be able to use it) but perhaps you can work with the data afterwards and convert that into some useable form.
It's actually written in Java, so it should run on anything with the JVM.
Oooh! Sounds awesome. :cool:
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community and follow The Piazza's Twitter feed so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.

User avatar
willpell
Black Dragon
Posts: 2896
Joined: Wed Sep 23, 2015 9:10 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by willpell » Fri Oct 16, 2015 2:52 am

Big Mac wrote:
willpell wrote:Does anyone on this board care enough about Magic to take care of those forums in this way?
I would strongly advise you to get in touch with M:tG communities as soon as possible. There are only 15 days left. You could perhaps act as a middleman, helping them to get a copy of Ender's app, but don't wait for him. If he gets into technical difficulties and runs out of time, they might not have time to come up with their own solution. The sooner you can get the word out, the more chance there is that a M:tG fan can come up with an automated solution to save their content and get it hosted on a M:tG or CCG fansite.
Any chance that you can link me to your portal into that site? I doubt I can ever find it again. By "portal", I mean wherever you first posted the "please come join us" ad (if you posted more than one, please link me to all of them, or as many as you can find - my Google-Fu would be terrible even if the Wotco boards had a functional internal search or could be effectively reached by Google advance searching, neither of which is true). I saw such an ad once, and that is how I am here; the page is the best possible rendezvous site thereon, but I will probably never find it, unless someone, most likely you as the creator of it, know exactly where it is.

User avatar
Falconer
Troll
Posts: 402
Joined: Wed Sep 22, 2010 10:42 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by Falconer » Fri Oct 16, 2015 3:45 am

There’s a software called Interarchy which I haven’t used for years (and don’t currently own), but which is (or at least used to be) able to crawl an entire website (eg.: community.wizards.com) and download everything from it.

thorr-kan
Ogre
Posts: 208
Joined: Thu Sep 16, 2010 7:40 pm

Re: Please archive content from the forums at WotC!

Post by thorr-kan » Fri Oct 16, 2015 5:23 pm

Big Mac wrote:
thorr-kan wrote:There is a concerted effort to archive the forums and the old d20 content being made through the http://www.giantitp.com forums.
Please would you post the link to the place/topic where this is being discussed/organised.
Main index seems to be:
http://www.giantitp.com/forums/showthre ... rds-forums

NB: I haven't been following it too closely, but the thread's active.

User avatar
willpell
Black Dragon
Posts: 2896
Joined: Wed Sep 23, 2015 9:10 pm
Gender: male

Re: Please archive content from the forums at WotC!

Post by willpell » Sat Oct 17, 2015 5:22 pm

Falconer wrote:There’s a software called Interarchy which I haven’t used for years (and don’t currently own), but which is (or at least used to be) able to crawl an entire website (eg.: community.wizards.com) and download everything from it.
Given that "http://www.community.wizards.com" or "http://www.wizards.com/community" doesn't go anywhere, I wouldn't count on this working. I don't really know how this aspect of computer science works, but I've observed in my day-to-day how easy it is for a link to break.

Post Reply

Return to “The Wishing Well”