00:32 <abhinavsingh> hi i would like to know can i set custom riak meta headers using erlang pb client?? or am i bound to use http interface for such tasks?

00:35 <abhinavsingh> here is the code snippet i have been trying (posted yesterday night too)

00:35 <abhinavsingh> http://pastebin.com/ymY6Kx5r

00:35 <abhinavsingh> attempt is to set custom meta headers while PUT of a new key inside bucket

00:36 <abhinavsingh> seems to work with "curl -X PUT -d '' -H ""...." methodology

00:36 <abhinavsingh> but not with riak pb client interface

00:43 <abhinavsingh> is thr any other place i can get some answers to my queries?? riak blog, basho blog or will stackoverflow be a good place?

00:49 <joseph_sh> what kind of error was it?

00:49 <joseph_sh> when you tried through pb interface?

00:52 <joseph_sh> does util:utf8 in your code produce binary?

00:55 <abhinavsingh> well thr are no error

00:55 <abhinavsingh> i can see my bucket/key

00:55 <abhinavsingh> set appropriately

00:56 <abhinavsingh> just that the custom headers are missing

00:56 <abhinavsingh> util:utf8() is basically standard implementation of utf8

00:56 <abhinavsingh> conversions which we see in erlang

00:57 <abhinavsingh> my util:utf8 is working fine for all other PUT's

00:57 <abhinavsingh> of mine

00:58 <abhinavsingh> just that in this case i want to set custom http headers and it fails here

00:58 <abhinavsingh> even adding Links while PUT works fine for me (just custom header step is broken for some reason)....

00:59 <joseph_sh> I'm not sure it works through the riak pb interface. It doesn't seem to accept custom headers, I thought it did

01:01 <joseph_sh> in riakc_pb.erl in the client:

01:01 <joseph_sh> pbify_rpbcontent_entry(_Key, _Value, PbContent) ->

01:01 <joseph_sh> %% Ignore unknown metadata - need to add to RpbContent if it needs to make it

01:01 <joseph_sh> %% to/from the client

01:01 <joseph_sh> PbContent.

01:09 <abhinavsingh> hmmm

01:09 <abhinavsingh> this is really bad for me... it means using http interface is my only last resort

01:09 <joseph_sh> actually

01:09 <joseph_sh> no

01:10 <joseph_sh> sorry

01:10 <joseph_sh> let me check something

01:10 <joseph_sh> but should be able to set your custom meta like so:

01:11 <joseph_sh> dict:store(<<"X-Riak-Meta">>, [{<<"TTL">>, TTL}, {<<"Status">>, Status}], M0)

01:12 <joseph_sh> the X-Riak-Meta in the pb client accepts a list of key pairs, which makes sense because that's how the protobuf message is structured

01:15 <joseph_sh> dict:to_list(riakc_obj:get_metadata(Obj2)).

01:15 <joseph_sh> [{<<"X-Riak-VTag">>,"5MkkeIaXixXS8eL2l8EOGZ"},

01:15 <joseph_sh> {<<"X-Riak-Last-Modified">>,{1327,644847,605759}},

01:15 <joseph_sh> {<<"X-Riak-Meta">>,[{"Test","testy"}]}]

01:15 <abhinavsingh> o i c.....well indeed

01:16 <abhinavsingh> i just chked this out and now i can see my TTL and Status headers present as

01:16 <abhinavsingh> Status created

01:16 <abhinavsingh> TTL 60

01:16 <abhinavsingh> in response headers.....

01:17 <abhinavsingh> thanks for the help Joseph, this should get me going as of now.....

01:17 <joseph_sh> no problem

01:18 <abhinavsingh> i think basho can put up some more help/demo/examples on the wiki for users like me....

01:18 <joseph_sh> I thought there was some stuff about that, but they only seemed to talk about it in reference to the http api, as far as I could tell

01:19 <joseph_sh> there are a lot of little things though

01:19 <joseph_sh> if there isn't anyone who can answer your question, the mailing list is a good way to contact people

01:20 <abhinavsingh> ohk... i actually subscribed for riak mailing list

01:20 <joseph_sh> especially at this time as most people who are on here often are in the US

01:20 <abhinavsingh> but i never got the email confirmation

01:20 <abhinavsingh> and as a result i think i m not on list

01:20 <abhinavsingh> will try again today

01:20 <joseph_sh> right, well, you can also check through something like riak.markmail.org

01:20 <abhinavsingh> cool thanks

01:20 <abhinavsingh> can understand

01:20 <abhinavsingh> i m in India and it's 11:50 AM

01:21 <abhinavsingh> i can imagine US time right now :)

01:21 <joseph_sh> yeah, I'm in Shanghai, so it's 2:20pm :D

01:21 <abhinavsingh> :)

01:21 <joseph_sh> yeah, it's 1:20am east coast I think

01:22 <DeadZen> yup

01:23 <DeadZen> i been up since 5am... i think thats it for a days work..

01:23 <joseph_sh> heh

01:23 <DeadZen> ;p

01:23 <joseph_sh> also, for the meta: member:abhinavsingh {<<"X-Riak-Meta">>,[{"X-Riak-Meta-Test2","wut"}]}

01:24 <joseph_sh> looks like you need the full key, oops

01:24 <joseph_sh> DeadZen, I thought you just never slept?

01:24 <DeadZen> not if i can help it.

01:27 <abhinavsingh> Yes i exactly did the same

01:27 <abhinavsingh> dict:store(<<"X-Riak-Meta">>,

01:27 <abhinavsingh> [{<<"X-Riak-Meta-CreatedTs">>, list_to_binary(float_to_list(util:ts()))},

01:27 <abhinavsingh> {<<"X-Riak-Meta-TTL">>, list_to_binary(integer_to_list(TTL))},

01:27 <abhinavsingh> {<<"X-Riak-Meta-Status">>, <<"created">>}], Meta0),

01:27 <abhinavsingh> and now i see my custom headers as expected

09:00 <cesarstafe> hi people

09:00 <cesarstafe> here again with a new question

09:02 <cesarstafe> how can I do to clean buckets? I mean, to delete all objects for example in 'users' or 'posts' or 'whatever' bucket?

09:02 <cesarstafe> can I do it from 'riak console' or some another place?

09:03 <seancribbs> cesarstafe: it's the same from either place

09:03 <seancribbs> list keys, delete them individually

09:05 <cesarstafe> @seancribbs: many thanks!

09:15 <loxs> is there any difference/pros/cons when using atoms instead of binaries for bucket names in riak_core?

09:18 <seancribbs> they will hash differently

09:18 <seancribbs> use binaries :)

09:38 <loxs> seancribbs, I can understand they will hash differently, but it doesn't matter (i guess) to me. Why use binaries?

09:38 <seancribbs> atoms consume memory that is never GCed

09:40 <DeadZen> atoms are like erlangs arterial plaque.

09:40 <DeadZen> or butter ;)

09:41 <seancribbs> yeah, if you have an external interface, an easy way to attack that system would be to create lots of buckets :)

09:42 <loxs> except if only usage of those buckets is internal. User can never specify the bucket explicitly

09:42 <DeadZen> loxs: if its under 1000 its fine..

09:43 <DeadZen> over that its turning out to be an uncontrolled resource leak,

09:43 <loxs> I guess it doesn't matter after all... binary or atom, so why not use binary...

09:43 <DeadZen> exactly

09:43 <loxs> (yes, it's a small number of buckets)

09:44 <DeadZen> in that case it doesn't matter, just remember atoms are permanent

09:44 <DeadZen> and there are no statin drugs ;)

09:44 <DeadZen> the vm will just run out of memory

09:44 <DeadZen> seancribbs: good morning

09:44 <seancribbs> ohai

09:44 <DeadZen> ;p

09:44 <DeadZen> seancribbs: got an odd question

09:45 <DeadZen> if a node downs in riak_core ring and requests are still going through

09:45 <DeadZen> sometimes you get back [] from preflist, randomly until that node comes back up or its removed

09:45 <DeadZen> is that normal?

09:46 <seancribbs> an unannotated preflist, yes

09:46 <DeadZen> is there a mitigation technique for dealing with that?

09:46 <seancribbs> use the active/annotated preflist

09:47 <DeadZen> is there an example?

09:47 <seancribbs> one sec, yeah

09:48 <seancribbs> https://github.com/basho/riak_kv/blob/master/src/riak_kv_put_fsm.erl#L180

09:49 <loxs> I have another question. Is there something like "riak_core plugin"?

09:49 <DeadZen> oh excellent, thank you

09:49 <DeadZen> loxs: lol a vnode?

09:49 <loxs> the reason I'm asking this is because I'll be having two different applications, potentially using the same ring (doesn't matter really). But they have almost nothing in common.

09:50 <seancribbs> make them both riak_core apps

09:50 <seancribbs> vnodes (if necessary)

09:50 <seancribbs> that's how we do kv/search/pipe

09:50 <loxs> right, but how do I configure riak_core twice in the same vm?

09:50 <seancribbs> you don't

09:51 <seancribbs> you just make your second app start up

09:51 <seancribbs> and it does the normal riak_core stuff like registering vnodes, announcing services, etc

09:58 <loxs> seancribbs, and it will spawn one vnode per partition for both of my apps? if so, great.

09:58 <seancribbs> yep

09:58 <loxs> I see, thanks

09:58 <seancribbs> whatever you register a vnode for, it will start up

09:58 <seancribbs> :)

09:59 <loxs> will there be any danger of bucket name clashes, etc.?

10:00 <loxs> or does it all depend on the name of the service (another namespace)

10:00 <seancribbs> each vnode can interpret the bucket differently

10:00 <seancribbs> in fact, they're not *required* to use buckets at all

10:00 <seancribbs> search doesn't, really

10:01 <seancribbs> neither does pipe

10:02 <loxs> hmm, I was abusing buckets till now. Using the bucket name as a "service" name (to distinguish what code to call)

10:02 <loxs> (in the common vnode, that knows of both apps)

10:04 <seancribbs> hmm that doesn't feel right

10:04 <seancribbs> yeah use a new vnode, they aren't difficult to write

10:04 <loxs> yes, I'll do what you suggested

10:04 <seancribbs> :)

10:04 <loxs> totally separate applications is best

10:04 <loxs> only awkward moment is that I need different ring settings for them

10:05 <loxs> (default n_val)

10:05 <seancribbs> that could be a place to use buckets :)

10:05 <loxs> yes :)

11:31 <loxs> does riak_core support sending commands to self (vnode) after some time?

11:32 <loxs> the equivalent of handle_info of a gen_server for example, where one can do send_after(self())

11:34 <marksteele> protocol buffers over ssl

11:34 <marksteele> possible?

11:36 <loxs> no, I'm talking about riak_core, not riak_kv (erlang stuff)

11:36 <seancribbs> loxs: use a timeout

11:36 <seancribbs> not send_after, generally

11:37 <loxs> seancribbs, what do you mean? and which callback will receive it?

11:38 <seancribbs> hold on

11:39 <marksteele> btw I assume it would be relatively easy to get riak to support ssl on the PB socket

11:39 <seancribbs> well i guess you could use send_after… but i thought there were timeouts in its replies

11:42 <seancribbs> hmm yeah send_after or send_command_after would be your best bet

11:47 <loxs> seancribbs, erlang:send_after? or there is some riak_core send_after?

11:48 <seancribbs> erlang

11:48 <loxs> which callback should receive it? handle_command?

11:50 <loxs> hmm, maybe handle_info, hope I won't break anything if I define my own handle_info?

11:50 <seancribbs> if you use send_command_after, handle_command will be invoked

11:51 <seancribbs> riak_kv uses erlang:send_after to do delayed deletes

11:51 <seancribbs> look at riak_kv_vnode sorta near the bottom

11:52 <loxs> yeah, that's exactly what I intend to do

15:17 <soapyillusions> quick question, how do we store a value with a Riak-defined key using https://github.com/basho/riak-java-client (storing in a bucket without specifing a key)