thayen/forgejo-thayen-dev
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

Revise and check robots.txt #1

New issue
Open
opened 2026-07-07 09:32:35 +00:00 by thayen · 0 comments
thayen commented 2026-07-07 09:32:35 +00:00
Owner
Copy link

It would be good to review and if necessary update the crawling rules for forgejo using a robots.txt file

Examples of existing robots files can be found at:
https://codeberg.org/robots.txt
https://codefloe.com/robots.txt (seems to be the default)

Some resources can be found linked in https://developer.mozilla.org/en-US/docs/Glossary/Robots.txt and https://moz.com/learn/seo/robotstxt

It would be good to review and if necessary update the crawling rules for forgejo using a `robots.txt` file Examples of existing robots files can be found at: `https://codeberg.org/robots.txt` `https://codefloe.com/robots.txt` (seems to be the default) Some resources can be found linked in `https://developer.mozilla.org/en-US/docs/Glossary/Robots.txt` and `https://moz.com/learn/seo/robotstxt`
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v1.0.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
thayen/forgejo-thayen-dev#1
No description provided.