News
news 1950s Science Kit Had Real Uranium
news We're Probably Living in a Simulation, Elon Musk Says
news It’s Now Possible To Telepathically Communicate with a Drone Swarm
news New material to 'speed spacecraft to 134,000,000 mph'
news The future of food: what we’ll eat in 2028
news Descend Into England’s Network of Secret Nuclear Bunkers
news British UFO hunter’s laptop was ‘wiped’ after mysterious death
news Expert warns satellites could be hacked and used as giant 'radio frequency weapons'
news 1,000-year-old Maya document that tracks the movement of Venus is real
news Google Maps' strangest objects revealed
news The Russian Family That Cut Itself Off From Civilization for More Than 40 Years


Username:
Password: or Register
 
Thread Rating:
  • 2 Vote(s) - 1 Average
  • 1
  • 2
  • 3
  • 4
  • 5

Here's the Deleted Thread That Shows You How to View Your Deleted Threads
killedbydeath
Conspirator
User ID: 384275
01-03-2018 07:45 PM

Posts: 111



Post: #16
RE: Here's the Deleted Thread That Shows You How to View Your Deleted Threads
Advertisement
not all threads are cached clearly

[Image: images?q=tbn:ANd9GcTypHsdtGwiNfI40N6CUXU...gRsRNiptKA]
Quote this message in a reply
Ghost of Skippy
lop guest
User ID: 97427
01-03-2018 07:48 PM

 



Post: #17
RE: Here's the Deleted Thread That Shows You How to View Your Deleted Threads
LoP Guest  Wrote: (01-03-2018 07:39 PM)
Skippy, why not just join properly and be the funny and playful poster you once were???



Quote this message in a reply
Adytum
I ♡ Israel
User ID: 415416
01-03-2018 08:02 PM

Posts: 2,106



Post: #18
RE: Here's the Deleted Thread That Shows You How to View Your Deleted Threads
Google only saves cached threads so long before the web scrawler tries to download the page again and save it.
Quote this message in a reply
NormalIsSubjective

User ID: 0
01-04-2018 12:44 AM

Posts: 3,434



Post: #19
RE: Here's the Deleted Thread That Shows You How to View Your Deleted Threads
In Like Flynn’  Wrote: (01-03-2018 07:37 PM)
NormalIsSubjective  Wrote: (01-03-2018 01:06 AM)
Is D still throwing the 'robots.txt' spanner in the Wayback Machine?

About /robots.txt
In a nutshell
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

It works likes this: a robot wants to vists a Web site URL, say http://www.example.com/welcome.html. Before it does so, it firsts checks for http://www.example.com/robots.txt, and finds:

User-agent: *
Disallow: /

The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

There are two important considerations when using /robots.txt:

robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.
So don't try to use /robots.txt to hide information.

See also:

Can I block just bad robots?
Why did this robot ignore my /robots.txt?
What are the security implications of /robots.txt?

The details <---LINK TO REST OF ARTICLE

Thanks for that.

I just wish D would let the Wayback Machine archive the site, then he could purge to his hearts content.
Quote this message in a reply








Contact UsConspiracy Forum. No reg. required! Return to TopReturn to ContentRSS Syndication