#chef

coderanger deed02392: Ruby's base64 library inserts newlines every 80 characters 00:18
It is moot because \n is ignored during base64 decode, but thats a thing 00:19
serversideup Hey all. Anyone having issues with the Apache2 cookbook? When I deploy it on a brand new Ubuntu 12.04 machine, I am getting this error: 01:27
[2013-01-26T19:23:47-06:00] FATAL: Could not load Moneta back end "BasicFile" 01:27
LoadError 01:28
--------- 01:28
cannot load such file -- chef/monkey_patches/moneta 01:28
coderanger serversideup: How did you install chef? 01:29
That was an issue with gem-based installs a few versions ago 01:29
serversideup I did this: sudo true && curl -L https://www.opscode.com/chef/install.sh | sudo bash 01:30
coderanger (moneta released a backwards-incompatible verison and our gemspec was too optimistic) 01:30
Okay, so you should have an omnibus install 01:30
serversideup My client is Chef 10.18.2 01:30
Here is my run list: https://dl.dropbox.com/u/13033256/RunList.png 01:32
coderanger Can you pastebin the full traceback? 01:32
serversideup http://pastebin.com/zWRGWJJh 01:34
coderanger serversideup: You need a -l debug to get the full error, or /srv/chef/file_store/chef-stacktrace.out 01:35
It looks like you have heavily customized your chef install though 01:37
just from the paths in use 01:37
So check that you aren't invoking moneta somewhere still 01:37
serversideup I didn't change any of the cookbooks. There were all default I believe 01:37
Here is the chef-stacktrace.out file : http://pastebin.com/jrMGh1D6 01:39
SHould I try downloading the cookbooks again and reuploading them? 01:39
coderanger That is the trace for a different exception 01:40
serversideup How should I run the debug command? I ran vagrant up to start (sorry, this is my first experience with Chef) 01:40
coderanger Are you using the vagrant provisioning stuff? 01:41
Because that might explain this, vagrant ships with an older version of Chef I think 01:41
the default baseboxes do anyway 01:41
You might have two different chef installs 01:42
but /srv/chef is definitely not a standard path 01:42
so something you did is very non-standard 01:42
what is the omnibus_updater cookbook? 01:43
serversideup I found that from one of the Chef community leaders on his tutorials. I thought it was for updating Chef 01:44
coderanger It looks like it is from heavywater, so I can't comment about if it does anything weird 01:45
but check if you have two chef-client installs 01:45
serversideup On my Macbook (the host)? For the guest, I created my own box and exported it with the "sudo true && curl -L https://www.opscode.com/chef/install.sh | sudo bash" command 01:46
Here is my Vagrant file: https://dl.dropbox.com/u/13033256/Vagrantfile.txt 01:47
jtimberman serversideup: we have published vagrant baseboxes that include chef 10.18.2: http://lists.opscode.com/sympa/arc/chef-dev/2013-01/msg00039.html 01:48
velosa Hi! I'm a chef/vagrant newbie. I'm trying to use the rvm::system recipe and it's stalling when 'Building rvm_ruby'. Not sure how to debug. 01:48
serversideup Thanks timber man, I'll try the same recipes and it and see if it fixes it 01:50
dyer Question, anyone here happen to be using the Ohai cookbook w/ Chef-Solo & Vagrant ? 01:51
coderanger serversideup: I would drop the omnibus_updater thing for now too 02:08
serversideup Thanks code ranger I will give that a try 02:08
serversideup coderanger: That omnibus_updater was the issue! Thanks for your help 02:28
It was trying to install an older version when I already had a newer one installed 02:29
coderanger :) 02:46
dyer Is there any way to force a reload of all the attributes in a chef run 03:36
dyer I am using the Ohai cookbook to add an ohai plugin but that plugin isnt loaded the first time until well after the attributes file are loaded 03:42
wondering if there is a way to restart the run somehow, or reload the attributes 03:42
jtimberman velosa: looks like it is a pretty hairy issue that fnichol is working to fix w/ rvm folks: https://github.com/fnichol/chef-rvm/issues/157#issuecomment-12747030 03:50
maek jtimberman: hi, hello. can you provide a possibly better way to do this " log("node[:celery][:version] must be set to use install_type 'source'") { level :fatal } if node[:celery][:version] == nil " ? 03:52
and is there a way I can make that log statement stop the chef run 03:52
jtimberman maek: use raise or Chef::Application.fatal! 03:52
maek right now it dies to the problem that missing attribute causes. 03:52
velosa jtimberman: Thanks! Didn't find that in my google searches. Looks like a fresh issue which probably explains why it was working a few weeks ago. 03:52
maek jtimberman: thanks! 03:53
jtimberman: is one 'better' then the other? 03:53
jtimberman maek: http://stackoverflow.com/questions/14290397/how-do-you-abort-end-a-chef-run/14294011#14294011 03:54
maek thank you google :( sorry. 03:54
jtimberman np 03:55
i just provided that answer recently :) 03:55
maek and what an answer, thank! 03:55
thanks, even. 03:56
jtimberman: Another approach is to use Chef::Application.fatal!. This logs a fatal message to the Chef logger and STDERR, and exits the application. You can also give it a return code (maybe you have a script that checks those?). can you elaborate on that? like if a script is watching the exit status of the chef-client to do x y or z based on exit code? 03:57
dyer jtimberman, question for you if you have a second I am trying to use your ohai cookbook to distribute a plugin and it doesnt seem to be working like I would expect it I am trying to get this plugin into context before attributes are loaded.. is that possible somehow ? 03:57
jtimberman maek: yeah probably. 04:07
:) 04:07
dyer: if you're distributing the plugin with chef, and want to use it in the same run you'd have to use the ohai resource to reload that plugin 04:08
dyer: our nginx cookbook does that in its ohai recipe 04:08
dyer isnt the ohai cookbook itself supposed to be doing that ? 04:08
jtimberman maybe.. 04:09
dyer so here is my problem.. I see it loading the plugin at the start of the chef run, but its well after all the chef attributes files have been loaded 04:10
jtimberman oh - yeah, you'd have to do it in a recipe, it won't be available in an attributes file. 04:11
dyer I am not sure I understand that last comment 04:11
jtimberman this is a case of "is it possible? yes. is it easy? probably not." 04:11
don't use the custom plugin's attributes in your cookbook's attributes file(s) to set attributes, do that in a recipe instead. 04:12
dyer oh, one more question 04:12
is there a default path for ohai plugins ? 04:12
jtimberman dyer: yeah... https://github.com/opscode/ohai/blob/master/lib/ohai/config.rb#L39 04:16
the plugins directory in the ohai installed location 04:17
dyer hrm 04:31
ok ty 04:31
I think I am going to submit a pull request to Mitchell to ohai_plugins_path in Vagrant 04:32
jtimberman is there ohai_plugins_path in chef itself? 04:32
that'd be handy to pass that off too 04:33
dyer you can put it in the the client.rb / solo.rb 04:33
Ohai::Config[:plugin_path] << "/somewhere" 04:33
jtimberman right 04:35
but, handy dsl helpers :) 04:35
dyer I am not sure I follow :( 04:35
jtimberman ohai_plugin_path is nicer to write than Ohai::Config[:plugin_path] 04:40
thinking from a non-Rubyist perspective 04:40
dyer ah, IC ur point 04:52
deed02392 anyone understand more the encrypted_data_bag_secret setting in knife.rb? 10:25
the docs aren't very clear about what exactly it does 10:25
zts deed02392: that sure is under-documented.... http://docs.opscode.com/chef/essentials_data_bags.html says that it'll be used by 'knife bootstrap', it's not clear whether the same option is used when encrypting/decrypting databags with knife 12:30
deed02392: it should definitely be documented here: http://docs.opscode.com/config_rb_knife.html 12:31
deed02392 indeed zts, I looked there already 12:32
as much as i'm not sure even what path to provide, the path that knife can read from so it copies it to the server? or the path that it will be copied to on the server it's bootstrapping 12:32
zts deed02392: the path that knife can read 12:36
deed02392: the location it will be copied to on the server comes from the bootstrap template 12:36
deed02392 ah all right zts 12:37
where's the bs template stored? 12:39
zts deed02392: it depends on how you've installed chef - the default ones are inside the gem 12:40
deed02392 i just cloned the default repos 12:41
zts deed02392: if you can find a file called 'ubuntu12.04-gems.erb', that will be in a directory with all the templates 12:41
deed02392 what's the gem? 12:41
zts deed02392: 'chef-full.erb' will be the default template for recent knifes though 12:41
deed02392: I think the gem is just called 'chef' 12:42
deed02392 oh i found it 12:43
it's in /opt/chef blah blah 12:43
it looks like chef expects you to use one secret for all your encrypted data bags 12:44
zts deed02392: that's certainly the common case. Search for "Access Encrypted Data from a Recipe" in http://docs.opscode.com/chef/essentials_data_bags.html for an example of specifying a non-default secret when loading a particular data-bag item 13:06
deed02392 gotcha zts, thanks! 13:06
zts no problem 13:07
zooz has anyone installed a chef server on fedora? 14:03
jondot` hey guys. do you recon logrotate being a deployment task or infra (chef) task? being that a certain server can host many services 15:55
asdmax hi, I have about 3k instances, chef server is very slow, how much cpu/memory should it have? Currently it runs on virtual server, with 6 cpu, 4gb ram and it uses only 1 chef-server-api thread (no wonder). 16:01
plan is to add more threads and add these behind nginx 16:01
do you guys have more suggestions to boost chef server performance? 16:03
kallistec asdmax: for now, you can run more chef-server-api instances with the default `thin` webserver, or run it with unicorn 16:18
Chef 11 server is waaaay faster and gives you concurrency out of the box with erlang 16:19
http://wiki.opscode.com/display/chef/Chef+11+Server+Preview 16:19
asdmax it's not production ready yet? 16:24
kallistec asdmax: beta, release probably in a week or so. No open bugs that I know of, but also no migration tools yet. 16:26
asdmax: so, depends on your tolerance for bleeding edge 16:27
jondot` kallistec: any blogpost about the various decision points in taking the erlang path (and pg/mysql) other than the 'introducing erchef' post? 16:27
asdmax ok, thanks 16:27
cheesepl_ jondot`: http://www.youtube.com/watch?v=FRFekZrLNas some is in the video 16:28
kallistec jondot`: no blog posts. Seth Falcon and Kevin Smith have done presentations 16:28
cheesepl_ ha, great timing 16:28
jondot` ah thanks, this should be awesome to see 16:29
cheesepl_ jondot`: my takeaway was couch didn't fit the usage patterns and Ruby/Merb is a memory hog ;) 16:29
kallistec I'm looking into rounding up some team members to do some "Chef 11 preview" blog posts similar to what we did for 0.10 16:29
cheesepl_ && jondot` pretty much, yeah. Couch doesn't work so well with write-heavy loads 16:30
cheesepl_ it also GROWS 16:31
jondot` i will keep my opinions of couch for myself. but i'll say that i *totally* understand any move off it. 16:31
I also see omnibus did a clj -> ruby move :) 16:31
kallistec as for ruby, some of the memory-hog stuff is because of how the app was originally written and coulda been fixed with a rewrite 16:32
and/or move to, say, JRuby 16:32
but erlang has lots of cool features for concurrency, such as per thread (erlang process/green thread) GC 16:33
so you don't get a situation where one big GC blocks everything, which you do with Java and therefore JRuby 16:34
cheesepl_ I already am biased toward erlang so I was really happy to see the talk 16:34
I'm* 16:34
jondot` kallistec: well, I believe the Java GC is pretty smart about stop-the-world collections, isn't it? 16:35
kallistec jondot`: you can change the algorithm and then tune it to mitigate the issue 16:36
jondot` yup. you do get a good amount of tunability 16:36
kallistec which involves learning about all the different algorithms and their tuning parameters 16:36
jondot` reminds me, this was a good read about it: http://www.manning.com/evans/ 16:37
kallistec for a dev/ops person who doesn't have production java in their infrastructure, it's quite a lot to ask for someone to run a chef server 16:38
jondot` so with erlang it was a fire-and-forget as far as you can tell? 16:41
kallistec as far as GC, yeah, we've never had issues with that locking the app 16:42
whereas we have that happen often enough with our Jenkins 16:42
which gets quite a lot fewer req/s 16:43
:P 16:43
cheesepl_ in my experience it scales with much, much less tuning 16:43
jondot` hmm. Jenkins always felt sluggish to me I wouldn't take it as a baseline for JVM apps 16:43
cheesepl_ and if you do need to tune it then you are really pushing it 16:43
jondot` well hopefull I can attempt reading erchef code soon :) 16:45
thanks for the chat! 16:45
kallistec The other thing is that JRuby perf really relies on invoke dynamic, which is only really good in the very newest JVM 16:45
so at the time we were looking at this, we couldn't necessarily rely on it 16:46
jondot` well I'm running JRuby in production for a while now. once JIT happens, it becomes much faster than MRI 16:46
I think indy is off the table for a while now. until Java 8 16:47
Truth is, I'm putting many 'traditional' Ruby infrastructure to shame with a mix of JRuby and Java interop :) 16:48
As we speak, i'm looking at my JRuby background workers, my answer for Resque -- 80MB, 20 workers, 144000 jobs per day and I can up it to 300 workers without blinking 16:50
kallistec for the latest java 7 VM, jruby will turn on indy by default 16:50
jondot` kallistec: did it change? I remember it was off by default, and you could enable with a flag - till Java 8 16:51
kallistec jondot`: I don't remember the details, just recall seeing that in a tweet from @headius 16:51
oliphox hi all - anybody have any idea why the rvm cookbook can't install any rubies? I just get "Failed to install rvm_ruby[ruby-1.9.3-p327]. Check logs in /log/ruby-1.9.3-p327" (and that log file doesn't exist) 16:54
jondot` hmm, mustve been very recent then 16:54
kallistec jondot`: anyway, as far as reading erlang code, you just need to wrap your head around pattern matching and figure out the record syntax 16:54
then it's pretty simple 16:54
jondot` kallistec: yep. with reading I'm OK, I suppose. Did a fair bit of PROLOG so it comes somewhat of a natural. Idioms and project workflow still a bit rough on me. 16:56
gotta hit the road. thanks, kallistec 16:56
kallistec you bet 16:57
oliphox: unfortunately, no. Only thing we do along those lines is install rbenv in our dev laptop cookbooks 16:58
if you don't get a response in a while you might hit up the mailing list 16:59
it gets pretty quiet in here on the weekends 16:59
oliphox kallistec: thanks for the replyi'll have a look at rbenv for now and see if i get any joy on the chef-rvm github tracker 17:00
gyre007 is there any way I can specify passowrd and username to Github when doing checkout via git resource from private repo ? 17:03
asdmax how big should be chef splay for +2k nodes? 17:05
kallistec asdmax: you'll have the load most evenly spread out if you set splay == interval 17:15
mal__ Evenin' all 18:03
Anyone got any pointers regarding the name attribute in metadata.rb? 18:04
carldanley as far as what goes? 18:04
mal__ I was hoping it allowed you to reference the cookbook via a different name 18:04
But it fails dependency checks, wondered if it was a bug or if I was using it wrong 18:05
ranjibd mal__, what version of chef you are using ? 18:06
mal__ i.e. folder name is cookbooks/app/ name attribute is 'company-app', then I wanted to do stuff like depends 'company-app' and include_recipe 'company-app' 18:06
client or server? 18:06
knife --version = Chef: 10.18.2 18:06
That was on my client; it's Chef: 10.14.2 on the server 18:07
ranjibd mal__, there was a discussion on the mailing list on this , i cant recall the conclusion though, let me search the tickets & mailing list 18:08
mal__ Gah, guess I need to upgrade the server, it's the standard version bootstrapped on ubuntu 11.04 18:08
ranjibd mal__, http://tickets.opscode.com/browse/CHEF-3307 18:09
mal__ ranjibd: I did find that, but it was 10.8 and there was no sort of discussion so I wondered if there was any more info 18:10
ranjibd i dont think its fixed 18:10
mal__ ranjibd: Fair enough, thanks for your help 18:11
poseid hm.. I am bit confused on how to get a new instance created with knife on AWS 18:56
not sure, where/how to use aws_access_key_id 18:56
kaerast poseid, I've not used aws but presumably it'll be in your ~/.chef/knife.rb file as with the Rackspace api key 19:05
poseid hmm.. I tried it there... but I get no valid AWS key... not sure, maybe an AWS problem 19:06
I'll likely go and check there 19:06
kaerast the ec2 bootstrap wiki page seems to cover everything: http://wiki.opscode.com/display/chef/EC2+Bootstrap+Fast+Start+Guide 19:07
poseid ah, great, the AWS settings screenshots look helpful 19:10
tolland i just setup a cookbook to generate a cron entry, and in my awesome wisdom tried this cron "do-this" do minute rand(60) 19:14
but presumably, thats resetting the minute value on every run 19:15
whats the idiom to generate a random number, and set it to the node? 19:15
chef having meetups like this 19:19
""Opscode Chef Meetups near Random Lake"" 19:19
is not helping for google results ;-) 19:19
kaerast I suppose you'd want to store the minute value as a node attribute really, and only set it if it isn't yet set 19:26
and/or you could use action create_if_missing so that it doesn't get updated on every run 19:27
although create_if_missing would require you to create it as a file in cron.d rather than using the cron resource 19:28
tolland the former sounds better, as id like to mess about with, and hence want to it updated 19:28
kaerast I'm not sure there's much in it really, but ok 19:30
carldanley sanity check 19:59
is this valid: notifies :reload, 'service[php5-fpm] service[nginx]' ? 19:59
perfabio hi all 20:07
carldanley quick question, if I wanted to say, import a directory and all .sql files contained within that folder how would I go about it? 20:12
without knowing the names first 20:12
perfabio sorry to bother, I'm having a weird problem using rackspace private cloud. It uses chef to manage hosts but seems that it's updating wrongly my ips. 20:14
I have 10nodes 10.100.0.11-21 called host01-11 ...hosts file is wrongly updated (by chef) pointing node11 to 10.100.0.60 instead of 10.100.0.21 ... so now, all my nodes can't work because the try to connect to 10.100.0.60 (my controller)!! any idea how to fix it? 20:15
perfabio anyone? 20:28
justincampbell perfabio: im guessing thats what the chef server knows about 20:39
if youre managing chef, take a look at the nodes on the server and their IPs 20:40
if you dont, contact rackspace support 20:40
perfabio can you point me how check nodes and ips? 20:41
maek perfabio: knife node show nodename -a ipaddress 20:41
or 20:41
knife node show nodename -a 20:41
perfabio thanks for the help 20:41
maek will show ALL nodes 20:41
er 20:41
all the nodes attributes 20:41
on cloud machines there are several attributes that have ips 20:41
also it would be worth looking in the cahced cookbook directory for the template thats generating /etc/hosts 20:42
to see what attribute its using 20:42
or the cookbook itself 20:42
if you have access 20:42
cached cookbook dir is usually /var/cache/chef/cookbooks 20:42
perfabio maek thanks! seems that ip is right: ipaddress: 10.100.0.21 20:43
maek id do this 20:44
knife search node "role:something_to_identify_these_10_boxes" -a |grep 10.100.0.60 20:45
if I understand the problem 20:45
that might help you see where that ip is coming from 20:45
perfabio maek that ip is nowhere! 20:48
maek are you sure your search string works? 20:49
if you dont |grep 20:49
do you see a metric butt load of json? 20:49
perfabio: ^? 20:49
seems like time to use that fanatical support rackspace boasts about :D 20:50
perfabio 10.100.0.60 is the ip of a virtual machine...so it should have no roles... (maybe) 20:50
maek well I was just getting at that if you could find that ip somewhere it might help figure out the problem 20:51
but if its not an attribute on node11 20:51
perfabio I tried knife search node "role:single-compute" -a ipaddress 20:51
and I got a list of all my nodes with the right ips 20:51
maek leave off ipaddress 20:51
mah 20:52
never mind 20:52
call rackspace ;) 20:52
perfabio without -a I get an error 20:52
... in `parse_options': missing argument: -a (OptionParser::MissingArgument) ... 20:53
but you are right...nevermind! 20:53
maek hrm. -a is supposed to give you all the node attributes 20:53
if you dont pass one for it to show 20:53
perfabio maek where can I find the template of /etc/hosts ? 20:54
or how can I disable this recipe? 20:54
maek perfabio: i would look in /var/cache/chef/cookbooks 20:54
find . -name hosts.erb 20:54
assuming thats how it works 20:54
perfabio no files founded! 20:55
maybe in /opt/rpcs/chef-cookbooks/cookbooks/osops-utils 20:55
maek yeah im just guessing as to where it would be located by default 20:56
ive never used rackspace 20:56
perfabio tnx maek, I'll try to solve it 21:05
maek can you do this 21:06
mv that /etc/hosts file 21:06
then run chef-clien in whyrun 21:06
and see what recipes would modify /etc/hosts? 21:06
perfabio: np, good luck. 21:06
perfabio maek sorry again, where can I find this log Chef::Log.info(...) ? 21:12
maek perfabio: Not sure what you mean 21:12
that seems to be log level info method for chef-client output 21:12
buzz 21:12
perfabio sorry but my english is very bad := 21:12
maek is ok 21:12
perfabio ah ok... 21:13
tnx 21:13
maek Chef::Log.info(...) as far as I know is used to send output messages during chef-client at the log level warn 21:13
can you run chef-client on node11? 21:13
perfabio yes, I'm trying 21:14
maek chef-client -l debug > /tmp/chef-client.out 21:14
and then dig through there for /etc/hosts 21:14
and it should show what recipe is changing it 21:14
perfabio hosts = search(:node, "chef_environment:#{node.chef_environment}") ...this should be the problem 21:17
ops ...maybe later !!! debugging mode 21:17
maek perfabio: thats just saying to grab all the nodes in the environment of the current node 21:17
that doesnt make sense as to why node11 is getting a different ip 21:18
perfabio http://pastebin.com/tWbyfUeh 21:19
it retrieves the wrong ip! 21:19
so I think that this is the real problem: ::Chef::Recipe::IPManagement.get_ip_for_net 21:21
maek but its working for the other nodes right? 21:22
perfabio yes, I'll explain what the problem is: I have physical nodes with ips 10.100.0.11 - 10.100.0.21 21:24
openstack can build VMs (with private ip) and per VM, I can assign a "public ip". My public ips are 10.100.0.54-10.100.0.254 21:25
this IPs are called floatting ips. 21:26
/etc/hosts assign to some physical node the right ip, and wrong to other, assigning floating IP!!! 21:27
iroller Ohai Chefs. Is there any good way go convert such things like GPG keys to encrypted data bags? 21:32
deed02392 what's the correct way to load an encrypted data bag and just have it use the default secret file? i.e. the one in /etc/chef/encrypted_data_bag_secret 22:43
should I just call Chef::EncryptedDataBagItem.load("bag", "item") and it'll use that secret by default? 22:44
mal__ Does anyone know how to mark a ticket as Fixed Provided? Do you have to own the ticket? 23:18