[security onion] kibana not start after soup on master and how to create new index and recovery dashboard configure.
https://discuss.elastic.co/t/error-kibana-server-is-not-ready-yet/156834/16###upgrade on master
sudo soup
reboot
sudo soup
reboot
so-stop
so-start
###Then kibana start fail
Waiting for ElasticSearch...connected!
so-kibana: WARN[0008] Error while downloading remote metadata, using cached timestamp - this might not be the latest version available remotely
bf01513e1ad2977c150a82dda6e3cda7110db0ff2283fdc08ba1f804e5a1ac41
##### Log => /var/log/kibana/kibana.log #####
{"type":"log","@timestamp":"2019-01-28T06:41:25Z","tags":["info","migrations"],"pid":1,"message":"Creating index .kibana_2."}
{"type":"error","@timestamp":"2019-01-28T06:41:25Z","tags":["fatal","root"],"pid":1,"level":"fatal","error":{"message":"[cluster_block_exception] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];","name":"Error","stack":"[cluster_block_exception] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]; :: {\"path\":\"/.kibana_2\",\"query\":
####Fixed by run in master node
curl -XPUT -H "Content-Type: application/json"
http://localhost:9200/.kibana_2-*/_settings -d '{"index.blocks.read_only_allow_delete": null}'
##or all index
curl -XPUT -H "Content-Type: application/json"
http://localhost:9200/.monitoring-*/_settings -d '{"index.blocks.read_only_allow_delete": null}'
##then
curl -XDELETE
http://localhost:9200/.kibanacurl -XDELETE
http://localhost:9200/.kibana_1curl -XDELETE
http://localhost:9200/.kibana_2##then
so-stop
so-start
and kibana could start [ok]
####Then Re-Create new index on kibana after delete .kibana index
In the UI, goto Management > Index Patterns and click Create Index Pattern.
The index pattern will be *:logstash-* and the time filter field name is "@timestamp".
####and running this script to re-create dashboard
https://github.com/Security-Onion-Solutions/securityonion-elastic/blob/master/usr/sbin/so-elastic-configure-kibana-dashboardscd /usr/sbin
vi so-elastic-configure-kibana-dashboards
##and running script
./bash so-elastic-configure-kibana-dashboards
then check dashboard come back again !
