Improve performance replacing os.listdir() with os.scandir()

[Paolo Melchiorre]

Use os.scandir() instead of os.listdir() in the remaining occurrences in the code:
​https://github.com/search?q=repo%3Adjango%2Fdjango+os.listdir&type=code

Based on the ​Python documentation

Using scandir() instead of listdir() can significantly increase the performance of code that also needs file type or file attribute information, because os.DirEntry objects expose this information if the operating system provides it when scanning a directory.

[Sarah Boyce]

Similar to #29689 accepting, thank you
Note that additional benchmarks in ​django-asv are always welcome 👍

[Tim Graham]

The description makes it sound like this is a simple find and replace all, however, do all usages "also need file type or file attribute information"?

[Amir Karimi]

Replying to Tim Graham:

The description makes it sound like this is a simple find and replace all, however, do all usages "also need file type or file attribute information"?

Good point! Except this case: ​https://github.com/django/django/blob/aa74c4083e047473ac385753e047e075e8f04890/scripts/manage_translations.py#L42
I didn't find any other cases where file attributes (is_dir, etc) are needed, and only their names or the number of list_dir output are needed. The only edge that "scandir" may still have is its less memory consumption when it comes to large folders (which I suspect is the case in any of these usages)

[Marcus Vinicius Araujo]

Replying to Amir Karimi:

Amir, my bad. I'm new to contributing to this project, and I didn't notice that this ticket had already been assigned to you.

Anyway, I submitted a patch.

[Amir Karimi]

Replying to Marcus Vinicius Araujo:

Replying to Amir Karimi:

Amir, my bad. I'm new to contributing to this project, and I didn't notice that this ticket had already been assigned to you.

Anyway, I submitted a patch.

That's fine. I forgot I had such a ticket. Next time, you can first ask what's going on with the task and if you don't hear back, you can assign it to yourself.

[Marcus Vinicius Araujo]

Replying to Amir Karimi:

Replying to Marcus Vinicius Araujo:

Replying to Amir Karimi:

Amir, my bad. I'm new to contributing to this project, and I didn't notice that this ticket had already been assigned to you.

Anyway, I submitted a patch.

That's fine. I forgot I had such a ticket. Next time, you can first ask what's going on with the task and if you don't hear back, you can assign it to yourself.

Will do! Thanks.

[Natalia Bidart]

After reviewing this ticket, the relevant documentation, and the PR, I remain unconvinced of the benefits of this change. Given Tim's question and answer (specifically, that only manage_translations.py requires file type or file attribute information) the proposed change seems to add an extra level of indentation to the code without a clear benefit.

Basic performance tests show only minimal improvements. In accordance with our ​current docs for requesting performance optimizations I think we need to mark this as wontfix until a clear and substantial performance gain is demonstrated. The risk of breaking systems that rely on os.listdir (for example, via monkeypatching) outweighs the potential unproven benefits.