Opened 10 years ago

Closed 9 years ago

#2634 closed defect (wontfix)

[search-api] Lucene issues with UTF

Reported by: Scater Owned by: nobody
Component: Contrib apps Version: other branch
Severity: normal Keywords: search-api; beck; lucene; index;
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


Here - information about adding search-api branch for Brian Beck's SoC project.

I have 2 major bugs in Lucene backend (

  1. No index in no-utf encoding. My database and Django-site in cp-1251 (Russian-Ukrainian). So i have 'InvalidArgsError' bug when i call indexer.update() method.

Exception Value: (, 'init', ('annotation', '\xe0\xe0\xe0\xe0\xe0\xe0\xe0\xe0\xe0', , ))
\xe0..\xe0 - this is my russian text in Annotation field

  1. When i complete update index in UTF, and use search like this:

for hit in


And update index once again - i have Java Exception: 'Cant delete file' or 'cant create file'

Change History (7)

comment:1 Changed 10 years ago by anonymous

  • milestone Version 0.93 deleted
  • Version 0.95 deleted

comment:2 Changed 10 years ago by anonymous

  • Owner bbeck deleted

comment:3 Changed 10 years ago by adrian

  • priority changed from high to normal
  • Severity changed from major to normal

comment:4 Changed 10 years ago by Michael Radziej <mir@…>

  • Version set to other branch

comment:5 Changed 10 years ago by Simon G. <dev@…>

  • Summary changed from Bugs in Brian Beck's SoC project. Search-API to [search-api] Lucene issues with UTF
  • Triage Stage changed from Unreviewed to Accepted

comment:6 Changed 9 years ago by jelle

I agree that this is an major show stopper for this otherwise very nice addition.
Its really a pity that Unicode isn't supported, that's such a severe error that it renders this effort pretty useless I'm afraid...
Which is really too bad, since it sure has a huge potential!

Here's a traceback:

In [4]: models.indexer.update()

<type 'exceptions.UnicodeEncodeError'> Traceback (most recent call last)

c:\Jelle_prive\Jelle_dev\workspace\JakobMacfarlane\src\jm_book_site\<ipython console> in <module>()

c:\Python25\lib\site-packages\django\contrib\search\ in update(self, documents)

53 for document in update_queue:
54 self.delete(document)

---> 55 self.index(document)

57 if close:

c:\Python25\lib\site-packages\django\contrib\search\ in index(self, row)

102 # newlines solves this.
103 contents = '\n'.join([str(getattr(row, for field in \

--> 104 self.text_fields])

105 # FIXME: Hardcoded 'contents' field.
106 document.add(PyLucene.Field('contents', contents,

<type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\u2019' in position 224: ordinal not in r

In [5]:

comment:7 Changed 9 years ago by Simon G. <dev@…>

  • Resolution set to wontfix
  • Status changed from new to closed

Currently the search-api is dead in the water, so I'm marking this as wontfix unless someone takes over the search-api

Note: See TracTickets for help on using tickets.
Back to Top