What is the purpose of the 'robots.txt' file on a website? MCQ with Answer and Explanation

Question
What is the purpose of the 'robots.txt' file on a website?
A. To configure the web server
B. To block malicious users
C. To instruct search engine crawlers which parts of the site to index
D. To store user login credentials
Answer: Option C
Solution (By JKSSB Mock Tests)
The robots.txt file is a standard used by websites to communicate with web crawlers, indicating which pages or sections should not be crawled or indexed.
This question belongs to: Computer Internet and Web Technologies
Request complete Subject PDF

Discuss this Question (0)

No comments yet. Be the first to start the discussion!

Practice More Internet and Web Technologies Questions

Question #1
What status code is returned by a web server to indicate a '301' redirection?
A. Bad Gateway
B. Moved Permanently
C. Unauthorized Access
D. Request Timeout
Discuss
Get Complete Subject PDF for Computer

Correct Answer: Option B

Explanation:
A 301 status code signifies a permanent redirection, indicating that the requested resource has been assigned a new permanent URI.
This question belongs to: Computer Internet and Web Technologies
Question #2
Which of the following ports is designated by default for secure, encrypted Post Office Protocol version 3 (POP3S) communication?
A. 443
B. 110
C. 995
D. 993
Discuss
Get Complete Subject PDF for Computer

Correct Answer: Option C

Explanation:
Port 995 is the globally recognized default network port assigned for POP3 operations run securely over an encrypted SSL/TLS connection.
This question belongs to: Computer Internet and Web Technologies
Question #3
Which of the following is a client-side storage mechanism in web browsers?
A. localStorage
B. Cookies
C. All of the above
D. sessionStorage
Discuss
Get Complete Subject PDF for Computer

Correct Answer: Option C

Explanation:
localStorage, sessionStorage, and cookies are all ways to store data on the client side, with different scopes and persistence.
This question belongs to: Computer Internet and Web Technologies
Explore All Internet and Web Technologies MCQs
Today's Top Topics
Computer Abbreviations 369 Memory and Storage Devices 438 Computer Fundamentals 326 Programming Languages 0 Operating Systems 510
Explore Subjects
Maths 5575 English 15164 Science 4430 Static General Knowledge 6362 Economy GK 2220
Practice Mix
Milk and water are in ratio 4:6 in a 273 L mixture. Quantity of milk?
The blaze ______ they doused was huge.
The 'Rani of Jhansi' is associated with:
The posters were ________ all over the city.
The principal that gives Rs. 453600 interest in 8 years at 9% SI is