Hello everyone, I came across an JSON error “JSON::NestingError: nesting of 1000 is too deep” when executing ‘sudo chef-client’ to setup a ubuntu-10-04-server-i386 machine as a hadoop name node. I ran cluster_chef on the my chef workstation node to launch and bootstrap the ubuntu node, then get the same error log reported when executing ‘sudo chef-client’ on the node it self. For debugging, I only add 2 “recipes [recipe[cluster_chef::dedicated_server_tuning], recipe[hadoop_cluster::namenode]] ” for the node.
Did anyone encounter this kind of issue and give some tips? Thanks in advance!!
———– ENV of the Ubuntun Node ———–
Linux ubuntu-vm 2.6.32-33-generic-pae #70-Ubuntu SMP Thu Jul 7 22:51:12 UTC 2011 i686 GNU/Linux ruby 1.8.7
(2010-01-10 patchlevel 249)
[i486-linux] json (1.6.1) chef
(0.10.8) ohai (0.6.10) Cookbook
hadoop_cluster version is 0.9.7
Cookbook hadoop version is 0.8.1
Cookbook cluster_service_discovery is 0.1
—————-ERROR LOG ———–
ubuntu@ubuntu-vm:~$ sudo chef-client
[Fri, 27 Jan 2012 15:32:32 +0800] INFO: *** Chef 0.10.8 ***
[Fri, 27 Jan 2012 15:32:36 +0800] INFO: Run List is [recipe[cluster_chef::dedicated_server_tuning], recipe[hadoop_cluster::namenode]]
[Fri, 27 Jan 2012 15:32:36 +0800] INFO: Run List expands to [cluster_chef::dedicated_server_tuning, hadoop_cluster::namenode]
[Fri, 27 Jan 2012 15:32:36 +0800] INFO: Starting Chef Run for demohadoop-master-0
[Fri, 27 Jan 2012 15:32:36 +0800] INFO: Running start handlers
[Fri, 27 Jan 2012 15:32:36 +0800] INFO: Start handlers complete.
[Fri, 27 Jan 2012 15:32:39 +0800] INFO: Loading cookbooks [apt, aws, cluster_chef, cluster_service_discovery, ebs, hadoop_cluster, java, ubuntu]
[Fri, 27 Jan 2012 15:32:39 +0800] WARN: Missing gem 'right_aws'
[Fri, 27 Jan 2012 15:32:39 +0800] INFO: Couldn't set performance parameters from instance type, estimating from 1 cores and 1026096 ram
[Fri, 27 Jan 2012 15:32:39 +0800] INFO: Registering to provide demohadoop-namenode: {}
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: Running exception handlers
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: Report handler Chef::Handler::ErrorReport raised #<JSON::NestingError: nesting of 1000 is too deep>
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:501:in `to_json'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:501:in `to_json'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/json-1.6.1/lib/json/common.rb:278:in `generate'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/json-1.6.1/lib/json/common.rb:278:in `pretty_generate'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/json_compat.rb:48:in `to_json_pretty'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler/error_report.rb:27:in `report'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:218:in `run_report_unsafe'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:206:in `run_report_safely'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:112:in `run_exception_handlers'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:111:in `each'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:111:in `run_exception_handlers'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:119
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:110:in `call'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:110:in `run_failed'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:109:in `each'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:109:in `run_failed'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:170:in `run'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:239:in `run_application'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `loop'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `run_application'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application.rb:67:in `run'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/chef-client:26
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/bin/chef-client:19:in `load'
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: /usr/bin/chef-client:19
[Fri, 27 Jan 2012 15:32:39 +0800] ERROR: Exception handlers complete
[Fri, 27 Jan 2012 15:32:39 +0800] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[Fri, 27 Jan 2012 15:32:39 +0800] FATAL: JSON::NestingError: nesting of 1000 is too deep
ubuntu@ubuntu-vm:~$ cat /var/chef/cache/chef-stacktrace.out
Generated at Fri Jan 27 15:32:39 +0800 2012
JSON::NestingError: nesting of 1000 is too deep
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:501:in `to_json'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:501:in `to_json'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/json_compat.rb:44:in `to_json'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/rest.rb:219:in `api_request'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/rest.rb:130:in `put_rest'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:626:in `save'
/var/chef/cache/cookbooks/cluster_service_discovery/libraries/cluster_service_discovery.rb:73:in `provide_service'
/var/chef/cache/cookbooks/hadoop_cluster/recipes/namenode.rb:31:in `from_file'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/cookbook_version.rb:578:in `load_recipe'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/language_include_recipe.rb:40:in `include_recipe'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/language_include_recipe.rb:27:in `each'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/language_include_recipe.rb:27:in `include_recipe'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/run_context.rb:72:in `load'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/run_context.rb:69:in `each'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/run_context.rb:69:in `load'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:195:in `setup_run_context'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:159:in `run'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:239:in `run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `loop'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application.rb:67:in `run'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/chef-client:26
/usr/bin/chef-client:19:in `load'
/usr/bin/chef-client:19
------------------
After I set Chef::JSONCompat::JSON_MAX_NESTING to 0 in /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/json_compat.rb, and run “sudo chef-client”, it reports this error “SystemStackError: stack level too deep” :
—————-ERROR LOG ———–
ubuntu@ubuntu-vm:~$ sudo chef-client
[Fri, 27 Jan 2012 15:36:56 +0800] INFO: *** Chef 0.10.8 ***
[Fri, 27 Jan 2012 15:36:59 +0800] INFO: Run List is [recipe[cluster_chef::dedicated_server_tuning], recipe[hadoop_cluster::namenode]]
[Fri, 27 Jan 2012 15:36:59 +0800] INFO: Run List expands to [cluster_chef::dedicated_server_tuning, hadoop_cluster::namenode]
[Fri, 27 Jan 2012 15:36:59 +0800] INFO: Starting Chef Run for demohadoop-master-0
[Fri, 27 Jan 2012 15:36:59 +0800] INFO: Running start handlers
[Fri, 27 Jan 2012 15:36:59 +0800] INFO: Start handlers complete.
[Fri, 27 Jan 2012 15:37:02 +0800] INFO: Loading cookbooks [apt, aws, cluster_chef, cluster_service_discovery, ebs, hadoop_cluster, java, ubuntu]
[Fri, 27 Jan 2012 15:37:02 +0800] WARN: Missing gem 'right_aws'
[Fri, 27 Jan 2012 15:37:02 +0800] INFO: Couldn't set performance parameters from instance type, estimating from 1 cores and 1026096 ram
[Fri, 27 Jan 2012 15:37:02 +0800] INFO: Registering to provide demohadoop-namenode: {}
[Fri, 27 Jan 2012 15:37:02 +0800] ERROR: Running exception handlers
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: Report handler Chef::Handler::ErrorReport raised #<SystemStackError: stack level too deep>
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/params_validate.rb:85:in `set_or_return'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:230:in `chef_environment'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:491:in `to_json'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:501:in `to_json'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/json-1.6.1/lib/json/common.rb:278:in `generate'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/json-1.6.1/lib/json/common.rb:278:in `pretty_generate'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/json_compat.rb:48:in `to_json_pretty'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler/error_report.rb:27:in `report'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:218:in `run_report_unsafe'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:206:in `run_report_safely'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:112:in `run_exception_handlers'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:111:in `each'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:111:in `run_exception_handlers'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/handler.rb:119
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:110:in `call'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:110:in `run_failed'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:109:in `each'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:109:in `run_failed'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:170:in `run'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:239:in `run_application'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `loop'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `run_application'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application.rb:67:in `run'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/chef-client:26
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/bin/chef-client:19:in `load'
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: /usr/bin/chef-client:19
[Fri, 27 Jan 2012 15:37:03 +0800] ERROR: Exception handlers complete
[Fri, 27 Jan 2012 15:37:03 +0800] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[Fri, 27 Jan 2012 15:37:03 +0800] FATAL: SystemStackError: stack level too deep
ubuntu@ubuntu-vm:~$ cat /var/chef/cache/chef-stacktrace.out
Generated at Fri Jan 27 15:37:03 +0800 2012
SystemStackError: stack level too deep
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/params_validate.rb:86:in `set_or_return'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:230:in `chef_environment'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:491:in `to_json'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:501:in `to_json'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/json_compat.rb:44:in `to_json'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/rest.rb:219:in `api_request'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/rest.rb:130:in `put_rest'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/node.rb:626:in `save'
/var/chef/cache/cookbooks/cluster_service_discovery/libraries/cluster_service_discovery.rb:73:in `provide_service'
/var/chef/cache/cookbooks/hadoop_cluster/recipes/namenode.rb:31:in `from_file'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/cookbook_version.rb:578:in `load_recipe'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/language_include_recipe.rb:40:in `include_recipe'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/language_include_recipe.rb:27:in `each'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/mixin/language_include_recipe.rb:27:in `include_recipe'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/run_context.rb:72:in `load'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/run_context.rb:69:in `each'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/run_context.rb:69:in `load'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:195:in `setup_run_context'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/client.rb:159:in `run'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:239:in `run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `loop'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application/client.rb:229:in `run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/application.rb:67:in `run'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/chef-client:26
/usr/bin/chef-client:19:in `load'
--------------------