Please archive content from the forums at WotC!

Industry news, rumoured and announced new products, and discussion of games company policies.

Moderator: Chimpman

Please archive content from the forums at WotC!

Postby Giant Space Hamster » Sat Sep 26, 2015 1:48 pm

The Piazza just made an open invitation to all Dungeons & Dragons fans. Now we are making a call to arms. Please help copy content from the various forums for campaign settings and rules at WotC to the forums for the same campaign settings and rules at The Piazza.

Topics where people can discuss archiving content for specific D&D campaign settings (deciding what to save, etc):

Topics where people can discuss archiving content for other things (deciding what to save, etc):

How to archive WotC forum content at The Piazza:
  1. Register for an account at The Piazza (if you do not already have an account here),
  2. Use one browser tab/window to navigate to a specific forum at the WotC forums that you would like to save content from,
  3. Use a second browser tab/window to navigate to the specific forum at The Piazza forums that you would like to move the content to,
  4. Copy the thread title from the forum at WotC,
  5. Use the "New Topic" button at the forum at The Piazza and paste in the topic title,
  6. Copy the first post from the WotC forum:
    • If you have an account at WotC, you can use the quote button, copy the post and then paste it into your topic here
    • If you do not have an account at WotC, you can highlight the text of the post, copy it and then paste it into your topic here
  7. You need to repeat the above two steps for every reply in the topic*.
  8. Check the WotC forum posts for links and special formatting and add them back in here, if you can**.

* = If you copy each post separately, that will give the forum admins the ability to "give the post back" to the person who made it, if they want to get their post back If you copy multiple WotC posts into a single Piazza post, then the post will stay as your post.
** = If you have trouble with formatting, you can ask for help here.

This post may be edited, to add in more information, as other specific archive attempts get made.
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
This account is only used for forum administration. For general contact, please use my personal user account.
Junior admin at The Piazza. My moderator voice is green.
User avatar
Giant Space Hamster
Initiate
 
Posts: 298
Joined: Tue Jun 07, 2011 8:55 pm
Location: London, UK

Re: Please archive content from the forums at WotC!

Postby Canageek » Wed Sep 30, 2015 6:25 am

This seems like a far to large undertaking to pull off in time; Has anyone looked into an automated solution that we could untangle later?
User avatar
Canageek
Hobgoblin
 
Posts: 45
Joined: Thu Sep 04, 2008 3:41 am
Location: Canada

Re: Please archive content from the forums at WotC!

Postby religon » Wed Sep 30, 2015 3:06 pm

I think ENWorld has an automated tool (PC's but not Macs). It might be easier to save to their 'lifeboat' boards.
User avatar
religon
Bugbear
 
Posts: 127
Joined: Mon Nov 26, 2012 9:09 pm

Re: Please archive content from the forums at WotC!

Postby Big Mac » Thu Oct 01, 2015 12:09 am

Canageek wrote:This seems like a far to large undertaking to pull off in time; Has anyone looked into an automated solution that we could untangle later?


I don't have the ability to make that sort of thing, but if you can look for someone who might be able to help, that would be great.

Meanwhile, I have started to rescue Spelljammer topics and will move onto another campaign setting, if I get that finished.

I've not decided which one to go for, but given that The Piazza has the world's most active Mystara community, I think that the Mystara fans should have the ability to take care of that setting.

Archiving all of Forgotten Realms and/or Eberron will probably be impossible for me, in the timescale. I might go for a smaller campaign setting, just so that I can have a chance to archive all of it.
David "Big Mac" Shepheard
Newsflash!: The Piazza is moving!
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.
User avatar
Big Mac
Giant Space Hamster
 
Posts: 21589
Joined: Sun Jun 15, 2008 3:52 pm
Location: London UK

Re: Please archive content from the forums at WotC!

Postby Big Mac » Thu Oct 01, 2015 12:15 am

religon wrote:I think ENWorld has an automated tool (PC's but not Macs). It might be easier to save to their 'lifeboat' boards.


It is only a semi-automated too. You still have to do some of the work.

I believe that MerricB's tool generates BB Code that can be posted into any forum that supports BB Code (which is now pretty much anywhere except Wizards of the Coast.

You are free to post things in EN World, or anywhere else, but considering that The Piazza split off from the Wizards of the Coast forum, back in 2008, we would love it if the topics that many of the forum members here posted before The Piazza existed could be brought back to our community.
David "Big Mac" Shepheard
Newsflash!: The Piazza is moving!
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.
User avatar
Big Mac
Giant Space Hamster
 
Posts: 21589
Joined: Sun Jun 15, 2008 3:52 pm
Location: London UK

Re: Please archive content from the forums at WotC!

Postby willpell » Sun Oct 04, 2015 10:33 pm

"Just because everyone shouldn't be like me, doesn't mean no-one should. Sometimes it takes a loud, angry asshole, who's full of himself, to say things that kinder, gentler people wouldn't say."
--The Amazing Atheist

Salvaged links:
http://www.asmor.com/scripts/4eMagicIte ... easure.php

D&D character:
viewtopic.php?f=75&t=17559&p=195819#p195819
User avatar
willpell
White Dragon
 
Posts: 2601
Joined: Wed Sep 23, 2015 9:10 pm

Re: Please archive content from the forums at WotC!

Postby Big Mac » Wed Oct 07, 2015 7:32 pm



No idea, I'm afraid. I know I have not done them, as I am still working on archiving Spelljammer topics.

If you use the forum search system, you could search for the names of the topics, to see if they are here. Maybe you could start archiving one or more of these topics yourself. :)
David "Big Mac" Shepheard
Newsflash!: The Piazza is moving!
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.
User avatar
Big Mac
Giant Space Hamster
 
Posts: 21589
Joined: Sun Jun 15, 2008 3:52 pm
Location: London UK

Re: Please archive content from the forums at WotC!

Postby willpell » Thu Oct 08, 2015 1:02 am

Big Mac wrote:If you use the forum search system, you could search for the names of the topics, to see if they are here. Maybe you could start archiving one or more of these topics yourself. :)


I can see about it, but I'm not going to have a lot of time this month. Had Wizards announced an October closure back in January, I would have had something to do with myself when I was content to be bored out of my skull, slowly consuming my savings account. Now I actually have to work, and a month is nowhere close to long enough for this amount of archiving.
"Just because everyone shouldn't be like me, doesn't mean no-one should. Sometimes it takes a loud, angry asshole, who's full of himself, to say things that kinder, gentler people wouldn't say."
--The Amazing Atheist

Salvaged links:
http://www.asmor.com/scripts/4eMagicIte ... easure.php

D&D character:
viewtopic.php?f=75&t=17559&p=195819#p195819
User avatar
willpell
White Dragon
 
Posts: 2601
Joined: Wed Sep 23, 2015 9:10 pm

Re: Please archive content from the forums at WotC!

Postby Big Mac » Thu Oct 08, 2015 3:11 am

willpell wrote:
Big Mac wrote:If you use the forum search system, you could search for the names of the topics, to see if they are here. Maybe you could start archiving one or more of these topics yourself. :)


I can see about it, but I'm not going to have a lot of time this month. Had Wizards announced an October closure back in January, I would have had something to do with myself when I was content to be bored out of my skull, slowly consuming my savings account. Now I actually have to work, and a month is nowhere close to long enough for this amount of archiving.


I have the same issue. I don't have much time. And I'm having connectivity problems and I'm not well. (I would be asleep right now if I hadn't been kept up by coughing.)

So I'm going to try to finish archiving Spelljammer before October 29th and then see if I can move onto trying to save some of something else.

I put the word out, as much as I could a couple of weeks earlier, but I've not got the time to co-ordinate things and also to save things. So I think we are all going to need to save what we can and if some stuff is lost, then it is lost.

But if you know of any other people, who might have more time than you, perhaps you can see if they have the time to save these topics that you think are worth saving. I've pretty much told everyone I know, but I'm sure there are some D&D fans out there who are totally unaware the WotC forums are due to be shut down in just over 20 days. :)
David "Big Mac" Shepheard
Newsflash!: The Piazza is moving!
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.
User avatar
Big Mac
Giant Space Hamster
 
Posts: 21589
Joined: Sun Jun 15, 2008 3:52 pm
Location: London UK

Re: Please archive content from the forums at WotC!

Postby Sturm » Thu Oct 08, 2015 8:58 am

The best would be to save the whole forum with an offline viewer. I'm trying something of the sort with the Other Worlds subforum now. I have no idea however if that's possible and what kind of size the final result will have.
Also, even if I manage to do that, it will be difficult or impossible to give access to it to others.
It's just another time in which Wizards has shown scarce consideration toward its customers. They could have simply close the forums and keep them online for reference, it would have not been such a big effort to do that.
Anyway I think all the Mystara material has already been archived in the Vaults, so I'm not concerned about it. Do not know what's the status of other world's material however.
User avatar
Sturm
Black Dragon
 
Posts: 3692
Joined: Sat Feb 12, 2011 7:26 pm
Location: Genoa, Italy

Re: Please archive content from the forums at WotC!

Postby willpell » Thu Oct 08, 2015 9:08 pm

Sturm wrote:The best would be to save the whole forum with an offline viewer. I'm trying something of the sort with the Other Worlds subforum now. I have no idea however if that's possible and what kind of size the final result will have.
Also, even if I manage to do that, it will be difficult or impossible to give access to it to others.
It's just another time in which Wizards has shown scarce consideration toward its customers. They could have simply close the forums and keep them online for reference, it would have not been such a big effort to do that.
Anyway I think all the Mystara material has already been archived in the Vaults, so I'm not concerned about it. Do not know what's the status of other world's material however.


Does anybody on this board intend to save any of the MTG material? A lot of it goes obsolete once a format rotates, but there's also some really cool creative stuff that's difficult to so much as locate.
"Just because everyone shouldn't be like me, doesn't mean no-one should. Sometimes it takes a loud, angry asshole, who's full of himself, to say things that kinder, gentler people wouldn't say."
--The Amazing Atheist

Salvaged links:
http://www.asmor.com/scripts/4eMagicIte ... easure.php

D&D character:
viewtopic.php?f=75&t=17559&p=195819#p195819
User avatar
willpell
White Dragon
 
Posts: 2601
Joined: Wed Sep 23, 2015 9:10 pm

Re: Please archive content from the forums at WotC!

Postby Gravesgard » Fri Oct 09, 2015 6:03 am

I've been saving a ton of ravenloft pages, html and files. So they can be accessed offline. I'll post them up when I have some more free time.
User avatar
Gravesgard
Goblin
 
Posts: 10
Joined: Fri Oct 02, 2015 11:38 pm

Re: Please archive content from the forums at WotC!

Postby Giant Space Hamster » Fri Oct 09, 2015 7:20 am

willpell wrote:Does anybody on this board intend to save any of the MTG material? A lot of it goes obsolete once a format rotates, but there's also some really cool creative stuff that's difficult to so much as locate.


I can not speak for Ashtagon, but this forum has never been a CCG forum. It's core has been to be a D&D forum, for campaign settings and rules.

The Piazza has grown well beyond it's original remit, but it has grown to cover non D&D campaign settings and rules - not to cover CCGs and other forms of gaming. So I do not think there a fanbase here that could facilitate a rescue attempt in the timescale. (If we had been prewarned of the WotC forum shutdown several years back, Ashtagon may have considered making a home for M:tG fans, and we may have been in a position to mobilise a large number of CCG fans. But that didn't happen, so I personally do not think that there is anyone here who can take on the vast workload of saving that content.)

If you wish to save M:tG material, I would strongly suggest you sign up to an existing M:tG forum, where they do not have any sort of need to save the D&D side of the WotC forums, and where they can focus 100 percent on saving the M:tG forum content. You have less than 20 days to organise this.

Good luck!
David "Big Mac" Shepheard
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
This account is only used for forum administration. For general contact, please use my personal user account.
Junior admin at The Piazza. My moderator voice is green.
User avatar
Giant Space Hamster
Initiate
 
Posts: 298
Joined: Tue Jun 07, 2011 8:55 pm
Location: London, UK

Re: Please archive content from the forums at WotC!

Postby Angel Tarragon » Fri Oct 09, 2015 5:41 pm

Gravesgard wrote:I've been saving a ton of ravenloft pages, html and files. So they can be accessed offline. I'll post them up when I have some more free time.
I have archived some of the older threads, I intend do more today, maybe even get it completely done.
User avatar
Angel Tarragon
Dawn Dragon
 
Posts: 8234
Joined: Sun Jun 06, 2010 2:39 am
Location: Malathéa

Re: Please archive content from the forums at WotC!

Postby thorr-kan » Fri Oct 09, 2015 6:50 pm

There is a concerted effort to archive the forums and the old d20 content being made through the www.giantitp.com forums.
thorr-kan
Bugbear
 
Posts: 178
Joined: Thu Sep 16, 2010 7:40 pm

Re: Please archive content from the forums at WotC!

Postby enderxenocide0 » Mon Oct 12, 2015 12:56 pm

I pooled together the collective knowledge of my friends and I in the software dev business and we're currently testing an application that lets us go through a forum and save all the content to a local hard drive, making it fully browsable offline. Once we're sure it works properly in the next few days, we'll share it with the community so anyone can back up a forum in full. It won't have search capabilities unless someone makes one for it, since Wizards is unlikely to release their database to us. What you do with the content once it's stored locally is up to you. If nothing else, it gives us an infinite amount of time to move content to new forums. We would be no longer constrained by time. I'll keep you all posted on the progress.
User avatar
enderxenocide0
Stone Giant
 
Posts: 692
Joined: Thu Dec 18, 2014 2:44 am

Re: Please archive content from the forums at WotC!

Postby willpell » Mon Oct 12, 2015 3:24 pm

enderxenocide0 wrote:I pooled together the collective knowledge of my friends and I in the software dev business and we're currently testing an application that lets us go through a forum and save all the content to a local hard drive, making it fully browsable offline. Once we're sure it works properly in the next few days, we'll share it with the community so anyone can back up a forum in full. It won't have search capabilities unless someone makes one for it, since Wizards is unlikely to release their database to us. What you do with the content once it's stored locally is up to you. If nothing else, it gives us an infinite amount of time to move content to new forums. We would be no longer constrained by time. I'll keep you all posted on the progress.


This sounds great. Sadly I have no home computer. Does anyone on this board care enough about Magic to take care of those forums in this way?
User avatar
willpell
White Dragon
 
Posts: 2601
Joined: Wed Sep 23, 2015 9:10 pm

Re: Please archive content from the forums at WotC!

Postby enderxenocide0 » Mon Oct 12, 2015 3:43 pm

If I get the time, I'll be happy to do it with any and every WotC forum (but don't necessarily rely on me as my priority will be Eberron first and I have limited time in the coming weeks).
User avatar
enderxenocide0
Stone Giant
 
Posts: 692
Joined: Thu Dec 18, 2014 2:44 am

Re: Please archive content from the forums at WotC!

Postby Big Mac » Wed Oct 14, 2015 12:49 am

thorr-kan wrote:There is a concerted effort to archive the forums and the old d20 content being made through the http://www.giantitp.com forums.


Please would you post the link to the place/topic where this is being discussed/organised.

enderxenocide0 wrote:I pooled together the collective knowledge of my friends and I in the software dev business and we're currently testing an application that lets us go through a forum and save all the content to a local hard drive, making it fully browsable offline. Once we're sure it works properly in the next few days, we'll share it with the community so anyone can back up a forum in full. It won't have search capabilities unless someone makes one for it, since Wizards is unlikely to release their database to us. What you do with the content once it's stored locally is up to you. If nothing else, it gives us an infinite amount of time to move content to new forums. We would be no longer constrained by time. I'll keep you all posted on the progress.


That's great.

I also have a friend on Facebook who is attempting the same sort of thing. He said he is trying to make something called MAFF files. Obviously both of you (and anyone else attempting it) only have a fixed amount of time to catch things before WotC pull down the original source.

I would suggest you do testing on a long multipage topic (like the Commoner Campaign topic) to see if your app can cope with something like that that is a bit weird.

I'm guessing that your app is probably going to be PC based (which will mean that I probably won't be able to use it) but perhaps you can work with the data afterwards and convert that into some useable form.

If you think you could make a fully hyperlinked browsable archive, perhaps we could talk to Ashtagon about the possibility of creating an archive on The Piazza. I would like to be able to import topics into The Piazza somehow (as I have been doing with the Spelljammer topics) as that would allow WotC conversations to be reactivated.

(I've been trying to do one Piazza post per WotC post, to give me the theoretical ability to give a post back to the original poster...assuming I can confirm that it is the same person. In theory that could mean that we could capture all of Keith Baker's Eberron topics/posts, with your app, eventually get them transferred to the Eberron forum at The Piazza and then eventually get them transferred over to his Piazza account, so that he could edit and update things, if necessary. But that would probably be something that got finished a year or more after the WotC forum was gone.)

willpell wrote:Does anyone on this board care enough about Magic to take care of those forums in this way?


I would strongly advise you to get in touch with M:tG communities as soon as possible. There are only 15 days left. You could perhaps act as a middleman, helping them to get a copy of Ender's app, but don't wait for him. If he gets into technical difficulties and runs out of time, they might not have time to come up with their own solution. The sooner you can get the word out, the more chance there is that a M:tG fan can come up with an automated solution to save their content and get it hosted on a M:tG or CCG fansite.

And if they are already ahead of us, perhaps you could return here with news of a fully working archive system. :)

enderxenocide0 wrote:If I get the time, I'll be happy to do it with any and every WotC forum (but don't necessarily rely on me as my priority will be Eberron first and I have limited time in the coming weeks).


If you can save the entire Eberron forum, that would be awesome. It is the second largest campaign setting forum at the WotC forums, and it would be impossible to grab it using the copy and paste technique I've used to save about 20 percent of the Spelljammer topics. ;(

You are doing a great thing. Thank you! If you are able to go beyond Eberron and save the rest of the campaign setting area...and then the entire D&D area...that would be a major contribution to D&D fandom. But I would expect you to look after Eberron first. :)

Keep up the good work. And good luck!
David "Big Mac" Shepheard
Newsflash!: The Piazza is moving!
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.
User avatar
Big Mac
Giant Space Hamster
 
Posts: 21589
Joined: Sun Jun 15, 2008 3:52 pm
Location: London UK

Re: Please archive content from the forums at WotC!

Postby enderxenocide0 » Wed Oct 14, 2015 4:31 am

Big Mac wrote:I'm guessing that your app is probably going to be PC based (which will mean that I probably won't be able to use it) but perhaps you can work with the data afterwards and convert that into some useable form.

It's actually written in Java, so it should run on anything with the JVM.
User avatar
enderxenocide0
Stone Giant
 
Posts: 692
Joined: Thu Dec 18, 2014 2:44 am

Re: Please archive content from the forums at WotC!

Postby Big Mac » Thu Oct 15, 2015 12:36 pm

enderxenocide0 wrote:
Big Mac wrote:I'm guessing that your app is probably going to be PC based (which will mean that I probably won't be able to use it) but perhaps you can work with the data afterwards and convert that into some useable form.

It's actually written in Java, so it should run on anything with the JVM.


Oooh! Sounds awesome. :cool:
David "Big Mac" Shepheard
Newsflash!: The Piazza is moving!
Please join The Piazza's Facebook group, The Piazza's Facebook page and The Piazza's Google + community so that you can stay in touch.
Spelljammer 3E Conversion Project - Spelljammer Wiki - The Spelljammer Image Group.
Moderator of the Spelljammer forum. My moderator voice is green.
User avatar
Big Mac
Giant Space Hamster
 
Posts: 21589
Joined: Sun Jun 15, 2008 3:52 pm
Location: London UK

Re: Please archive content from the forums at WotC!

Postby willpell » Fri Oct 16, 2015 2:52 am

Big Mac wrote:
willpell wrote:Does anyone on this board care enough about Magic to take care of those forums in this way?


I would strongly advise you to get in touch with M:tG communities as soon as possible. There are only 15 days left. You could perhaps act as a middleman, helping them to get a copy of Ender's app, but don't wait for him. If he gets into technical difficulties and runs out of time, they might not have time to come up with their own solution. The sooner you can get the word out, the more chance there is that a M:tG fan can come up with an automated solution to save their content and get it hosted on a M:tG or CCG fansite.


Any chance that you can link me to your portal into that site? I doubt I can ever find it again. By "portal", I mean wherever you first posted the "please come join us" ad (if you posted more than one, please link me to all of them, or as many as you can find - my Google-Fu would be terrible even if the Wotco boards had a functional internal search or could be effectively reached by Google advance searching, neither of which is true). I saw such an ad once, and that is how I am here; the page is the best possible rendezvous site thereon, but I will probably never find it, unless someone, most likely you as the creator of it, know exactly where it is.
User avatar
willpell
White Dragon
 
Posts: 2601
Joined: Wed Sep 23, 2015 9:10 pm

Re: Please archive content from the forums at WotC!

Postby Falconer » Fri Oct 16, 2015 3:45 am

There’s a software called Interarchy which I haven’t used for years (and don’t currently own), but which is (or at least used to be) able to crawl an entire website (eg.: community.wizards.com) and download everything from it.
User avatar
Falconer
Troll
 
Posts: 372
Joined: Wed Sep 22, 2010 10:42 pm

Re: Please archive content from the forums at WotC!

Postby thorr-kan » Fri Oct 16, 2015 5:23 pm

Big Mac wrote:
thorr-kan wrote:There is a concerted effort to archive the forums and the old d20 content being made through the http://www.giantitp.com forums.


Please would you post the link to the place/topic where this is being discussed/organised.

Main index seems to be:
http://www.giantitp.com/forums/showthre ... rds-forums

NB: I haven't been following it too closely, but the thread's active.
thorr-kan
Bugbear
 
Posts: 178
Joined: Thu Sep 16, 2010 7:40 pm

Re: Please archive content from the forums at WotC!

Postby willpell » Sat Oct 17, 2015 5:22 pm

Falconer wrote:There’s a software called Interarchy which I haven’t used for years (and don’t currently own), but which is (or at least used to be) able to crawl an entire website (eg.: community.wizards.com) and download everything from it.


Given that "http://www.community.wizards.com" or "http://www.wizards.com/community" doesn't go anywhere, I wouldn't count on this working. I don't really know how this aspect of computer science works, but I've observed in my day-to-day how easy it is for a link to break.
User avatar
willpell
White Dragon
 
Posts: 2601
Joined: Wed Sep 23, 2015 9:10 pm

Next

Return to The Wishing Well

Who is online

Users browsing this forum: No registered users and 5 guests