Opened 10 years ago

Closed 9 years ago

#2634 closed defect (wontfix)

[search-api] Lucene issues with UTF

Reported by: Scater Owned by: nobody
Component: Contrib apps Version: other branch
Severity: normal Keywords: search-api; beck; lucene; index;
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


Here - information about adding search-api branch for Brian Beck's SoC project.

I have 2 major bugs in Lucene backend (

  1. No index in no-utf encoding. My database and Django-site in cp-1251 (Russian-Ukrainian). So i have 'InvalidArgsError' bug when i call indexer.update() method.

Exception Value: (, 'init', ('annotation', '\xe0\xe0\xe0\xe0\xe0\xe0\xe0\xe0\xe0', , ))
\xe0..\xe0 - this is my russian text in Annotation field

  1. When i complete update index in UTF, and use search like this:

for hit in


And update index once again - i have Java Exception: 'Cant delete file' or 'cant create file'

Change History (7)

comment:1 Changed 10 years ago by anonymous

milestone: Version 0.93
Version: 0.95

comment:2 Changed 10 years ago by anonymous

Owner: Brian Beck deleted

comment:3 Changed 10 years ago by Adrian Holovaty

priority: highnormal
Severity: majornormal

comment:4 Changed 10 years ago by Michael Radziej <mir@…>

Version: other branch

comment:5 Changed 10 years ago by Simon G. <dev@…>

Summary: Bugs in Brian Beck's SoC project. Search-API[search-api] Lucene issues with UTF
Triage Stage: UnreviewedAccepted

comment:6 Changed 9 years ago by jelle

I agree that this is an major show stopper for this otherwise very nice addition.
Its really a pity that Unicode isn't supported, that's such a severe error that it renders this effort pretty useless I'm afraid...
Which is really too bad, since it sure has a huge potential!

Here's a traceback:

In [4]: models.indexer.update()

<type 'exceptions.UnicodeEncodeError'> Traceback (most recent call last)

c:\Jelle_prive\Jelle_dev\workspace\JakobMacfarlane\src\jm_book_site\<ipython console> in <module>()

c:\Python25\lib\site-packages\django\contrib\search\ in update(self, documents)

53 for document in update_queue:
54 self.delete(document)

---> 55 self.index(document)

57 if close:

c:\Python25\lib\site-packages\django\contrib\search\ in index(self, row)

102 # newlines solves this.
103 contents = '\n'.join([str(getattr(row, for field in \

--> 104 self.text_fields])

105 # FIXME: Hardcoded 'contents' field.
106 document.add(PyLucene.Field('contents', contents,

<type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\u2019' in position 224: ordinal not in r

In [5]:

comment:7 Changed 9 years ago by Simon G. <dev@…>

Resolution: wontfix
Status: newclosed

Currently the search-api is dead in the water, so I'm marking this as wontfix unless someone takes over the search-api

Note: See TracTickets for help on using tickets.
Back to Top