978

I want to remove all empty strings from a list of strings in python.

My idea looks like this:

while '' in str_list:
    str_list.remove('')

Is there any more pythonic way to do this?

Improve this question
5
  • 50
    @Ivo, neither of those statements are true. You should never modify a list that your iterating over using for x in list If you are using a while loop then it's fine. the loop demonstrated will remove empty strings until there are no more empty strings and then stop. I actually hadn't even looked at the question (just the title) but I answered with the exact same loop as a possibility! If you don't want to use comprehensions or filters for sake of memory, it's a very pythonic solution.
    – aaronasterling
    Oct 2, 2010 at 12:55
  • 4
    Still a very valid point to never change the list you're iterating over :)
    – Eduard Luca
    Feb 12, 2016 at 21:09
  • 1
    @EduardLuca if the point of iterating over a list is to change it, then that's the opposite of what you should do. You just have to be careful that you know that you do not cause an unexpected behavior by doing so.
    – jfa
    Apr 1, 2016 at 16:16
  • 1
    @EduardLuca, @JFA : The point is that he is NOT iterating over any list. He would if he had written something in the form for var in list:, but here, he has written while const in list:. which is not iterating over anything. it's just repeating the same code until a condition is false.
    – Camion
    Mar 19, 2019 at 23:26
  • 1
    You can use filter to remove the empty strings. The code should look something like this... data = list(filter(None, str_list))
    – Jacob Ward
    Dec 3, 2020 at 1:26

14 Answers 14

Reset to default
1546

I would use filter:

str_list = filter(None, str_list)
str_list = filter(bool, str_list)
str_list = filter(len, str_list)
str_list = filter(lambda item: item, str_list)

Python 3 returns an iterator from filter, so should be wrapped in a call to list()

str_list = list(filter(None, str_list))
Improve this answer
13
  • 19
    If you're that pressed for performance, itertool's ifilter is even faster—>>> timeit('filter(None, str_list)', 'str_list=["a"]*1000', number=100000) 2.3468542098999023; >>> timeit('itertools.ifilter(None, str_list)', 'str_list=["a"]*1000', number=100000) 0.04442191123962402.
    – Humphrey Bogart
    Jul 21, 2011 at 11:02
  • 4
    @cpburnz Very true. However, with ifilter results are evaluated lazily, not in one go—I'd argue that for most cases ifilter is better. Interesting that using filter is still faster than wrapping an ifilter in a list though.
    – Humphrey Bogart
    Sep 14, 2012 at 11:03
  • 8
    If you do this to a list of numbers, note that zeroes will also be removed (note: I only used the first 3 methods), so you'll need an alternate method.
    – SnoringFrog
    Apr 22, 2014 at 6:29
  • 3
    This focuses only on speed, not on how pythonic the solution is (the question that was asked). List Comprehensions are the pythonic solution, and filter should only be used if profiling has proven that the listcomp is a bottleneck.
    – Tritium21
    Feb 21, 2015 at 18:18
  • 3
    @whoever-mentions-about-or-imply-Python-3, please just edit and update the answer. We were only discussing for the Python 2 when this question was asked, even Python 3 was released almost 2 years. But do update both Python 2 and 3 results.
    – livibetter
    Mar 29, 2016 at 22:22
438

Using a list comprehension is the most Pythonic way:

>>> strings = ["first", "", "second"]
>>> [x for x in strings if x]
['first', 'second']

If the list must be modified in-place, because there are other references which must see the updated data, then use a slice assignment:

strings[:] = [x for x in strings if x]
Improve this answer
5
  • 42
    I like this solution because it's easily adaptable. If I needed to remove not only empty strings but strings that are just whitespace, for example: [x for x in strings if x.strip()].
    – Bond
    Dec 29, 2015 at 16:28
  • 1
    [x for x in strings if x] This works fine but Please explain how this loop is working??
    – Amar Kumar
    Jan 31, 2021 at 17:08
  • 8
    @AmarKumar In Python, blank strings evaluate to false when announced in a Boolean context, like in if x. The brackets, for loop, and if clause combine to read "generate a list that consists of x for every element in strings if x actually contains something." @Ib33x Absolutely awesome work. This answer is certainly the most Pythonic.
    – Nat Riddle
    Feb 22, 2021 at 21:37
  • Nice. [x for x in strings if x.strip()] to remove strings of white space.
    – PatrickT
    Aug 2, 2022 at 9:20
  • this doesnt filter nan
    – ss301
    Mar 20 at 17:35
110

filter actually has a special option for this:

filter(None, sequence)

It will filter out all elements that evaluate to False. No need to use an actual callable here such as bool, len and so on.

It's equally fast as map(bool, ...)

Improve this answer
2
  • 9
    This is a python idiom, in fact. It is also the only time I still use filter(), list comprehensions have taken over everywhere else.
    – kaleissin
    Feb 18, 2014 at 8:24
  • I find this easier to see the intention of the code, compared to a list comp
    – Martin CR
    Dec 17, 2020 at 13:10
35 
>>> lstr = ['hello', '', ' ', 'world', ' ']
>>> lstr
['hello', '', ' ', 'world', ' ']

>>> ' '.join(lstr).split()
['hello', 'world']

>>> filter(None, lstr)
['hello', ' ', 'world', ' ']

Compare time

>>> from timeit import timeit
>>> timeit('" ".join(lstr).split()', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
4.226747989654541
>>> timeit('filter(None, lstr)', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
3.0278358459472656

Notice that filter(None, lstr) does not remove empty strings with a space ' ', it only prunes away '' while ' '.join(lstr).split() removes both.

To use filter() with white space strings removed, it takes a lot more time:

>>> timeit('filter(None, [l.replace(" ", "") for l in lstr])', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
18.101892948150635
Improve this answer
3
  • it won't work if you have space among the string of a word. for example: ['hello world', ' ', 'hello', ' '] . >> ['helloworld', ' ', 'hello', ' '] do you have any other solution to keep spaces within an item in the list but removing others?
    – Reihan_amn
    Feb 6, 2018 at 19:06
  • 1
    Notice that filter(None, lstr) does not remove empty strings with a space ' ' Yeah, because that isn't an empty string.
    – AMC
    Jan 9, 2020 at 20:41
  • Lifesaver 🤓 !!
    – Abu Shoeb
    Jan 2, 2021 at 6:13
32

Sum up best answers:

1. Eliminate emtpties WITHOUT stripping:

That is, all-space strings are retained:

slist = list(filter(None, slist))

PROs:

  • simplest;
  • fastest (see benchmarks below).

2. To eliminate empties after stripping ...

2.a ... when strings do NOT contain spaces between words:

slist = ' '.join(slist).split()

PROs:

  • small code
  • fast (BUT not fastest with big datasets due to memory, contrary to what @paolo-melchiorre results)

2.b ... when strings contain spaces between words?

slist = list(filter(str.strip, slist))

PROs:

  • fastest;
  • understandability of the code.

Benchmarks on a 2018 machine:

## Build test-data
#
import random, string
nwords = 10000
maxlen = 30
null_ratio = 0.1
rnd = random.Random(0)                  # deterministic results
words = [' ' * rnd.randint(0, maxlen)
         if rnd.random() > (1 - null_ratio)
         else
         ''.join(random.choices(string.ascii_letters, k=rnd.randint(0, maxlen)))
         for _i in range(nwords)
        ]

## Test functions
#
def nostrip_filter(slist):
    return list(filter(None, slist))

def nostrip_comprehension(slist):
    return [s for s in slist if s]

def strip_filter(slist):
    return list(filter(str.strip, slist))

def strip_filter_map(slist): 
    return list(filter(None, map(str.strip, slist))) 

def strip_filter_comprehension(slist):  # waste memory
    return list(filter(None, [s.strip() for s in slist]))

def strip_filter_generator(slist):
    return list(filter(None, (s.strip() for s in slist)))

def strip_join_split(slist):  # words without(!) spaces
    return ' '.join(slist).split()

## Benchmarks
#
%timeit nostrip_filter(words)
142 µs ± 16.8 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

%timeit nostrip_comprehension(words)
263 µs ± 19.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter(words)
653 µs ± 37.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter_map(words)
642 µs ± 36 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter_comprehension(words)
693 µs ± 42.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter_generator(words)
750 µs ± 28.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_join_split(words)
796 µs ± 103 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
Improve this answer
2
  • s and s.strip() can be simplified to just s.strip().
    – AMC
    Jan 9, 2020 at 20:53
  • s and s.strip() is needed if we want to fully replicate filter(None, words), the accepted answer. I corrected x2 sample functions above and dropped x2 bad ones.
    – ankostis
    Jan 10, 2020 at 10:46
24

Reply from @Ib33X is awesome. If you want to remove every empty string, after stripped. you need to use the strip method too. Otherwise, it will return the empty string too if it has white spaces. Like, " " will be valid too for that answer. So, can be achieved by.

strings = ["first", "", "second ", " "]
[x.strip() for x in strings if x.strip()]

The answer for this will be ["first", "second"].
If you want to use filter method instead, you can do like
list(filter(lambda item: item.strip(), strings)). This is give the same result.

Improve this answer
1
  • Can you explain this piece? normally, x.strip() returns False, we have the result but how this code works I didn't understand the logic.
    – Fuad Ak
    Jun 17, 2022 at 7:13
18

Instead of if x, I would use if X != '' in order to just eliminate empty strings. Like this:

str_list = [x for x in str_list if x != '']

This will preserve None data type within your list. Also, in case your list has integers and 0 is one among them, it will also be preserved.

For example,

str_list = [None, '', 0, "Hi", '', "Hello"]
[x for x in str_list if x != '']
[None, 0, "Hi", "Hello"]
Improve this answer
6
  • 2
    If your lists have disparate types (except None), you may have a bigger problem.
    – Tritium21
    Feb 21, 2015 at 18:14
  • What types? I tried with int and other numeric types, strings, lists, tupes, sets and None and no problems there. I could see that if there are any user defined types that do not support str method might give a problem. Should I be worried about any other?
    – thiruvenkadam
    Feb 23, 2015 at 6:53
  • 1
    If you have a str_list = [None, '', 0, "Hi", '', "Hello"], it is a sign of a poorly designed application. You shouldn't have more than one interface (type) and None in the same list.
    – Tritium21
    Feb 23, 2015 at 16:21
  • 3
    Retrieving data from db? list of arguments for a function while doing automated testing?
    – thiruvenkadam
    Feb 24, 2015 at 5:22
  • 3
    Those are usually tuples.
    – Tritium21
    Feb 24, 2015 at 10:47
15

You can use something like this

test_list = [i for i in test_list if i]

where test_list is list from which you want to remove empty element.

Improve this answer
11

Depending on the size of your list, it may be most efficient if you use list.remove() rather than create a new list:

l = ["1", "", "3", ""]

while True:
  try:
    l.remove("")
  except ValueError:
    break

This has the advantage of not creating a new list, but the disadvantage of having to search from the beginning each time, although unlike using while '' in l as proposed above, it only requires searching once per occurrence of '' (there is certainly a way to keep the best of both methods, but it is more complicated).

Improve this answer
3
  • 1
    You can edit the list in place by doing ary[:] = [e for e in ary if e]. Much cleaner and doesn't use exceptions for control flow.
    – Krzysztof Karski
    Jun 28, 2017 at 12:01
  • 2
    Well, that's not really "in place" -- I'm pretty sure this creates a new list and just assigns it to the old one's name.
    – Andrew Jaffe
    Aug 13, 2018 at 9:22
  • This performs very poorly as the tail of data gets shuffled around in memory on each remove. Better to remove all in one hit.
    – wim
    Jan 9, 2020 at 22:27
10

As reported by Aziz Alto filter(None, lstr) does not remove empty strings with a space ' ' but if you are sure lstr contains only string you can use filter(str.strip, lstr)

>>> lstr = ['hello', '', ' ', 'world', ' ']
>>> lstr
['hello', '', ' ', 'world', ' ']
>>> ' '.join(lstr).split()
['hello', 'world']
>>> filter(str.strip, lstr)
['hello', 'world']

Compare time on my pc

>>> from timeit import timeit
>>> timeit('" ".join(lstr).split()', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
3.356455087661743
>>> timeit('filter(str.strip, lstr)', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
5.276503801345825

The fastest solution to remove '' and empty strings with a space ' ' remains ' '.join(lstr).split().

As reported in a comment the situation is different if your strings contain spaces.

>>> lstr = ['hello', '', ' ', 'world', '    ', 'see you']
>>> lstr
['hello', '', ' ', 'world', '    ', 'see you']
>>> ' '.join(lstr).split()
['hello', 'world', 'see', 'you']
>>> filter(str.strip, lstr)
['hello', 'world', 'see you']

You can see that filter(str.strip, lstr) preserve strings with spaces on it but ' '.join(lstr).split() will split this strings.

Improve this answer
2
  • 1
    This only works if your strings do not contain spaces. Otherwise, you're splitting those strings as well.
    – phillyslick
    Jul 25, 2017 at 13:32
  • 2
    @BenPolinsky as you reported join solution will split strings with space but filter will not. Thank you for you comment I improved my answer.
    – Paolo Melchiorre
    Aug 31, 2017 at 8:07
8

Keep in mind that if you want to keep the white spaces within a string, you may remove them unintentionally using some approaches. If you have this list

['hello world', ' ', '', 'hello'] what you may want ['hello world','hello']

first trim the list to convert any type of white space to empty string:

space_to_empty = [x.strip() for x in _text_list]

then remove empty string from them list

space_clean_list = [x for x in space_to_empty if x]
Improve this answer
2
  • 2
    if you want to keep the white spaces within a string, you may remove them unintentionally using some approaches. Like this approach, then?
    – AMC
    Jan 9, 2020 at 20:49
  • Thanks dude, it worked for me with a little change. i.e. space_clean_list = [x.strip() for x in y if x.strip()]
    – Muhammad Mehran Khan Attari
    Jan 26, 2020 at 8:56
6

Use filter:

newlist=filter(lambda x: len(x)>0, oldlist)

The drawbacks of using filter as pointed out is that it is slower than alternatives; also, lambda is usually costly.

Or you can go for the simplest and the most iterative of all:

# I am assuming listtext is the original list containing (possibly) empty items
for item in listtext:
    if item:
        newlist.append(str(item))
# You can remove str() based on the content of your original list

this is the most intuitive of the methods and does it in decent time.

Improve this answer
1
  • 9
    Welcome to SO. You have not been ignored. You have not been attacked by an anynonmous downvoter. You have been given feedback. Amplifying: Your proposed first arg for filter is worse than lambda x: len(x) which is worse than lambda x : x which is the worst of the 4 solutions in the selected answer. Correct functioning is preferred, but not sufficient. Hover your cursor over the downvote button: it says "This answer is not useful".
    – John Machin
    Jan 11, 2012 at 11:23
1

match using a regular expression and a filter

lstr = ['hello', '', ' ', 'world', ' ']
r=re.compile('^[A-Za-z0-9]+')
results=list(filter(r.match,lstr))
print(results)
Improve this answer
0

This is what worked for me. I had nan values as well since I created this list from a pandas dataframe column values.

[x for x in a if pd.notna(x) and x.strip()]
Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.