Rig is Offline and connection problems

edited February 26 in Networking
The common question is
My rig is mining, but it's offline in Hive

A bit of explanation. There is an agent on a rig that sends stats every 10 seconds to server. It start as soon as the system boots. So your rig maybe running but the agent fails to reach server for some reason.

Or the common problem is a dead filesystem and agent just can't save temp files to construct it's package.

For network diagnosis there is test script, just run
net-test
It will quickly ping and try to reach server. Take manual actions if you need more information.






Manual checking instructions below


First in all ensure your network connection.
- Check LEDs are blinking on the network card
- You have internet on the on the other devices in this net like you phone or laptop
- Try to open some site form the rig with the browser
- Check your connection by pinging some server, you should run "ping google.com" for instance
These are very basic actions you may take

The following is what I actually do on the particular rig when it has such problem
- "ping hiveos.farm" to check if hive server is reachable
- "mtr hiveos.farm" to check if the network has any packet loss
- "time curl http://hiveos.farm"; to check if HTTP ports are open and there is no significant lag

But the really-really first thing I really do is I open "agent-screen". It will show you what it tries to send and gives you some hint what's wrong. Like here
nvxqh2a34xub.jpg
There is obviously some problem with server connection, to investigate this you might do all the upper stuff)


Here is how the very bad situation with your internet looking. Very high packet loss.
iod0sqwp8nhm.jpg


This is a good command to test real times of opening the website. So you can see how much does it take to connect to server. If the value is high, like several seconds then you have a bad connection.
for i in {1..10}; do time curl -s https://hiveos.farm > /dev/null; done

To inspect the mirrors please try to "curl" api urls like
curl  http://api.hiveos.farm
curl  https://api.hiveos.farm
curl  http://amster.hiveos.farm
curl <...other urls listed in rig mirror selection>
«1

Comments

  • -- reserved --
  • встретил сегодня с аналогичной ситуацией, интернет в том месте по факту есть, но рига в панели сайта не видно, риг судя по SSH работает нормально, а на пуле по 0.
    К сожалению не было времени разбираться, поэтому просто попросил человека перезагрузить. Исходя из этого вопроса задал вопрос про возможность активации пинга на ватчдоге от опендев.
  • я разобрался в чем дело. нужно сделать следующее:
    на своем роутере установите резервацию по МАК адресу, для вашего неработающего рига... чтобы потом вы не подключали к нему монитор и т.д. и не искали его IP адрес у себя в сети.

    1. зайти в hive кабинет на сайте
    2. сбросить пароль на нужном риге на любой другой, при этом установить галочку изменить пароль только в базе, чтобы команд на риг не отправлялось.
    3. открываем браузер и коннектимся к своему ригу через локальную сеть: 192.168.1.15:4200
    4. login:user , pass:1
    5. вбиваем команду firstrun -f (тем самым сбрасываем все настройки на настройки по умолчанию)
    6. подтверждаем если требуется
    7. вбиваем логин рига зарегистрированный на сайте hive и тот самый новый пароль, который меняли только в базе hive.
  • Столкнулся с проблемой что иногда риг вешает локальную сеть и роутер.
    Роутер не пингуется с наружи. Внутри сети другие компьютеры не пингуются. Линки на сетевухах горят.
    Случалось с двумя разными ригами (по одному в сети) в разных местах
    Проявляется на роутерах Zyxel Extra 2, Dlink 855, Zyxel Giga 2, какой-то SNR.
    Пока не было возможности подключить монитор для локализации, может подскажете что куда копать, как диагностировать?
    Мать в обоих случайях Asrock h110pro BTC+
  • gaevsky ваше колдунство помогло !) так же и пинги отличные и связь есть и все хорошо, но риг был оффлан. Пробовал создавать новые риги с новыми паролями — ничего не помогало, а вот эти шаманские действия помогли :)
    версия хайва 05-19
  • please upgrade the e1000e module, because on certain motherboards with this integrated network card the actual driver version have an issue provoking all network crash not only this system your whole net.
    download the latest intel version https://downloadcenter.intel.com/download/15817

    Actual e1000e module of HiveOS 3.2
    Intel latest module : 3.4.0.2-NAPI

    regards
    Pablo
  • Всем привет. Аналогичная ситуация была с пропажей рига из онлайна, работает, майнит, а вот на сайте не виден. Постаянно срабатывал встроенный ватчдог. В моём случае майнил zcash в настройках указал телеметрию на сайт, как только её отключил, всё заработало. Видно нельзя указывать 2 телеметрии.
  • I'm having problems connecting a new rig based on the Asus z-270-p mobo today. The first rig based on the same mobo was effortless earlier today, but the second one is driving me nuts. I'm kind of getting desperate here, so if anyone else have any advice then I'd be really grateful - advance thanks!
  • Some of my rigs constantly switch between appearing offline/online. The rigs are working but I have no control over them unless I'm in the location to ssh into them. I've tried every server in the list and the same thing happens after a while. Ssh-ing into the miner and using the TIME CURL SERVER_ADDRESS command sometimes (not always) makes the rig appear online but it's VERY annoying not being able to rely on the information in the dashboard. Also the stats are affected by the rig appearing offline so stats are basically useless. Any solution for this issue?
  • So my rig just disconnected for some reason, can't get it back on. It was stuck on the watchdog reboot loop since then.

    Internet connection is fine

    ping hiveos.farm was fine with 50ms avg

    agent-screen returns "there is no screen to be attached matching agent. Starting new screen session."
    then I killed all dead screens using screen-wipe.
    agent-screen again and it returns same as above, creating a new screen sesion.
    screen -r returns "1907.agent ... (dead???)"

    What to do now?
  • Im having connection problem as well. Last night my rig ethernet port isn't lighting up anymore. On my rig account, its offline. I keep getting "Host name lookup failed." Ifconfig show eth0 with no IP. It was working so well until last night. I did have a couple morning where my rig goes down and all I have to do is restart miner. But last night my ethernet port doesn't even light up like it's dead completely. I had tried switching cable to my smart tv and it's working. Switched 2 different cable that I tested on other system that work but no luck. I turn the whole system off all night. This morning, I turn it back on and it work. Now, 2 hour later, I get a telegram message saying my rig is down again. Im at work now and wont be able to do anything until I get home 8 hour later. The ethernet I have is a rtl8111/8168. I remember last night I checked with dmeg | grep r816 and it shows r8169. I dont know if it a driver issue or what. Can anyone help?
  • I've been experiencing the issue of appearing offline in HiveOS while still mining.

    ping hiveos.farm comes back with time = 160
    mtr hiveos.farm comes back with two points of high packet loss, xe-9-3-0.edge has 74% packet loss while ae-2-3202.ear2 is experiencing 90% packet loss.

    How do we fix high packet loss?

    Any help is greatly appreciated!


  • edited February 20
    Hi, my agent-screen looks like this, rig is mining, but shown as offline. Worked perfectly for several weeks, then all of a sudden this happened. Plz help
    hscitqsfsgvd.png

    Tue Feb 20 21:02:38 MSK 2018
    /hive/sbin/gpu-stats: line 183: 24987 Segmentation fault jq -c -n --argjson temp "$jsontemps" --argjson fan "$jsonfans" --argjson load "$jsonload" --argjson power "$jsonpower" --argjson busids "$jsonbusids" --argjson brand "$jsonbrand" '{$temp, $fan, $load, $power, $busids, $brand}'
    Failed to read ewbf stats from localhost:42000
    Hashrate ewbf 0 kH/s
    /hive/bin/agent: line 277: 25009 Done echo $request
    25010 Illegal instruction | jq '.' -c
    Invalid response: http://amster.hiveos.farm
    {"jsonrpc": "2.0", "error": {"code": -32700, "message": "Parse error"}, "id": null}

  • wow, segmentation fault is something really wrong. Either the binary is corrupt (read - filsystem) or os-hardware-mem-unknown issue (which i don't believe).
  • got the same issue. the website is showing that the rig is offline.3h8ctd70nutw.jpg

    but when i check the agent screen and miner, everything is running. but with lower hashrate. all the other described stuff like ping and net-test are fine.
    2tucphleluwx.jpg

    any ideas?

    the only thing that helps, is to reconfigure the rig with firstrun -f. but that failure happens several times a day.
  • I have just moved my 14 rigs from Simple mining to HiveOS. I will also move my other 2 rigs. I have paid for the rigs hiveOs usage. HiveOs is a great for mining. Easy to use. But there is problem: all of the rigs are mining but seems offline on hiveos.farm web site. The internet connection is fine. No connection problem. I have double checked my internet connection. I found a solution.:

    -Power off the rig.
    -Power on the rig.
    -Type "hello" and hit the enter.
    -It fails because of limited connection time (after 4 seconds, it fails.)
    -Type "hello" and hit the enter.
    -It fails because of limited connection time (after 4 seconds, it fails.)
    -Power off the rig.
    -Power on the rig.
    -Type "hello" and hit the enter. You will see "Touching /tmp/.hive-hello-ok" message.
    -All of the rigs are online !

    Just follow these steps on a rig. No need to dothem on all of the rigs. Only one rig and BAM ! All rigs are online.

    But it s so strange that all of the rigs will go offline again in 20 minutes.

    @dev
    Please fix this problem. I need to check my rigs if I use the HiveOs. There is a time problem while connecting to your hive servers. HiveOs tries to reach amster.hiveos.farm
    is it possible to change the server ?
  • I have exactly the same problem. they are mining but with low speed and with an unknown wallet. I cant reach them with ssh or vnc. the only thing that helps is a manual restart but after about 20min, the same shit again.

    the longterm solution for me was to make new installations on USB sticks. since 3 days everything seems to be fine so far.

    @dev
    get back to me if you need any kind of logs.
  • all rigs are online and no problem now with the last realese
  • edited March 4
    Столкнулся сегодня с такой ошибкой:
    - пропало электричество — появилось
    - ферма перезагрузилась автоматом, всё ок
    - на сайте hiveos.farm ферма показывается как оффлайн, однако по факту работает и майнит

    Причина в том, что после перезагрузки роутера у фермы изменился DHCP IP, а вот на сайт он не подтянулся.

    Решение:
    - через роутер найти новый IP рига
    - изменить пароль в DB на сайте
    - запустить через https://192.168.X.X:4200/ firstrun -f и заново вбить Rig ID и новый пароль

    Риг опять онлайн в личном кабинете. 

    Как workaround проблемы вижу добавление ригов со статическими IP, чтобы такой ситуации с перебоями не повторилось, однако проблема имеет место быть по дефолту и рекомендуется к исправлению

    Спасибо!
  • Спонтанно риг вешает всю локальную сеть, роутер и даже домовое оборудование провайдера, при этом на ноуте ошибка DNS.

    Отключение рига от роутера поднимает сетку обратно, подключаю обратно, снова сеть падает, решается хард ресетом.

    История одинакова у разных провайдеров в разных квартирах и с разными роутерами, риги тоже соответственно разные за исключением матерей - ASRock h110 btc - в обоих случаях
  • Подскажите, что может быть не так. HiveOS не получает DNS серверы по DHCP. Роутер микротик раздает IP, клиенты под виндой и андроидом получают все как надо, с DNS. HiveOS получает IP, а DNS пустой.5a4ccyj9ttmg.jpg
    od9p53bfdlbs.jpg
  • DimaFern wrote: »
    The common question is
    My rig is mining, but it's offline in Hive

    A bit of explanation. There is an agent on a rig that sends stats every 10 seconds to server. It start as soon as the system boots. So your rig maybe running but the agent fails to reach server for some reason.

    - "mtr hiveos.farm" to check if the network has any packet loss

    Here is how the very bad situation with your internet looking. Very high packet loss.
    iod0sqwp8nhm.jpg

    Hello!

    I experience high packet loss, aprox. 90% on ae-2-3202.ear2.Paris1.Level3.net
    Agent-screen says curle_operation_timeout sending stats to paris.hiveos.farm.
    If this occures 3 times in a row, the rig goes offline. I can work with 3 GPUs in these conditions, but with 4 GPUs it often goes offline.

    What should I do? Please help me!

    Thank you very much,
  • I experience high packet loss, aprox. 90% on ae-2-3202.ear2.Paris1.Level3.net

    I have same problem and rigs are lot of time offline, but after 30sec-2 minutes come up again.
    Is it a bug, or there is a fix for it?3kjdexmdj9ea.png
  • I saw in the consol that.

    7jixlogvjr71.png

    Please help, I would like to fix it...
  • I've been going crazy trying to figure out what's going on with my rigs. I hope you all can help. Up to the last week my rigs have been working with no issues. After upgrading from .5-32 to .5-36 they started to show that they were offline. Checked them out and they were still mining. The problem started out as going offline a few times a day to just staying offline. I've checked the agent screen and I'm getting this error. Error sending status http://Paris.hiveos.farm. net-status also shows a connection failed to same address. I've checked my firewall, called the ISP (they said they are not blocking), downgraded to .5-32, did a fresh flash of the SSD of both .5-32 and .5-36. Nothing has worked. Any ideas what is going on?
  • Is this forum active and someone from the hiveOS is doing support here?
    Or we are on our own?
    I have read many threats here with no answer and help.
    Which is the best way to get in touch with hiveOS support?
  • edited March 26
    У меня ферма отпадала в оффлайн, майнинг прекращался. Сама ферма вроди как работала (нет монитора что бы проверить) и не перезагружалась. Попробывал совет gaevsky, посмотрю что будет. Проверил, не помогло.
    В агенте есть проблема как у nicorex. Error sending status http://Paris.hiveos.farm. Ферма по SSH не доступна. В этой же сети вторая ферма работает без сбоев.
  • nicorex, have you been able to make contact with anyone at hive regarding your issues? I've made posts of my problem in this forum, Telegram HiveOS chat, and on the HiveOS Facebook and have not got a response.
  • nicorex I solve it. You should change statistic server.z1kjqqmehdvw.png
    qbc5l1k09gu6.png
    1.png 102.7K
    2.png 133.3K
  • Thanks Stayer. I changed the server and I've been up with no issues for two hours now.
Sign In or Register to comment.