Vectorized reader exceptions

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Vectorized reader exceptions

Swapnil Shinde
Hi All
    I am not sure if this is random exception but this is what I have observed -
Create and load carbondata table from Spark dataframe - Without dictionary_include on two INT columns - Works fine. I can "select *" on it.
Create and load carbondata table from same Spark datafrmae - With dictionary_include on two INT columns - Throws NullPointerException when I run "select *".

Stacktrace throws error here -

I found similar error fixed in this PR but not available in 1.1.0 version -
I kind of tried adding above PR's changes with additional necessary changes to build successfully. (Not sure if it was a good idea)
If I load again and run "select *"  it throws different error -
Inline image 1

I am not much familiar with vectorized reader source code so not sure how to fix this. However, due to above error I have to load that dataset without creating dimensions on columns I need for filtration.
As always, your help is very much appreciated!